-- Logs begin at Wed 2022-07-20 18:35:51 UTC, end at Wed 2022-07-20 20:20:42 UTC. -- Jul 20 19:08:47.181448 dv-cinder-835384-16-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 20 19:08:47.192182 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 20 19:08:47.194262 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 20 19:08:47 2022] *** Jul 20 19:08:47.194262 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 20 19:08:47.194262 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 20 19:08:47.194262 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: nodename: dv-cinder-835384-16-iscsi Jul 20 19:08:47.194262 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: machine: x86_64 Jul 20 19:08:47.194262 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: clock source: unix Jul 20 19:08:47.194262 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: pcre jit disabled Jul 20 19:08:47.194262 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: detected number of CPU cores: 4 Jul 20 19:08:47.194262 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: current working directory: / Jul 20 19:08:47.194262 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: detected binary path: /usr/bin/uwsgi-core Jul 20 19:08:47.194787 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: your processes number limit is 47680 Jul 20 19:08:47.194787 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: your memory page size is 4096 bytes Jul 20 19:08:47.194787 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: detected max file descriptor number: 2048 Jul 20 19:08:47.194787 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: lock engine: pthread robust mutexes Jul 20 19:08:47.194787 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: thunder lock: enabled Jul 20 19:08:47.194787 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 20 19:08:47.194787 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: Python version: 3.8.10 (default, Jun 22 2022, 20:18:18) [GCC 9.4.0] Jul 20 19:08:47.240260 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: Python main interpreter initialized at 0x5653205ae9c0 Jul 20 19:08:47.240260 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: python threads support enabled Jul 20 19:08:47.240260 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: your server socket listen backlog is limited to 100 connections Jul 20 19:08:47.240260 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: your mercy for graceful operations on workers is 90 seconds Jul 20 19:08:47.240664 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: mapped 403077 bytes (393 KB) for 2 cores Jul 20 19:08:47.240740 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: *** Operational MODE: preforking *** Jul 20 19:08:47.241000 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: *** uWSGI is running in multiple interpreter mode *** Jul 20 19:08:47.241000 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: spawned uWSGI master process (pid: 80200) Jul 20 19:08:47.241240 dv-cinder-835384-16-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 20 19:08:47.242077 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: spawned uWSGI worker 1 (pid: 80210, cores: 1) Jul 20 19:08:47.242183 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: spawned uWSGI worker 2 (pid: 80211, cores: 1) Jul 20 19:08:47.243662 dv-cinder-835384-16-iscsi devstack@keystone.service[80200]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 20 19:08:52.825407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:08:52.825407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: from cryptography.utils import int_from_bytes Jul 20 19:08:52.826259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:08:52.826259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: from cryptography.utils import int_from_bytes Jul 20 19:08:52.826672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:08:52.826672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: from cryptography.utils import int_from_bytes Jul 20 19:08:52.826782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 20 19:08:52.826782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: from cryptography.utils import int_from_bytes Jul 20 19:08:52.876423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 19:08:52.876638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 19:08:52.876775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 20 19:08:52.876889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] command line args: [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 19:08:52.876970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 20 19:08:52.877057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 19:08:52.877137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] command line args: [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 20 19:08:52.877269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ================================================================================ {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 19:08:52.877408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 20 19:08:52.877496 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] admin_token = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.877633 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ================================================================================ {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 20 19:08:52.877726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] config_dir = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.877911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] admin_token = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.878039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.878123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] config_source = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.878194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] config_dir = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.878299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] debug = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.878449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.878552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.878742 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] config_source = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.878742 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] debug = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.878887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.878959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.879027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.880441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.880588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.880588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.880778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.880901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.880983 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.881098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.881179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_config_append = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.881303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.881379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.881379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_dir = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.881593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_config_append = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.881803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_file = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.881879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.881990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.882074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_dir = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.882074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_file = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.882250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.882369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.882445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.882603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.882681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.882681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.882868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.882982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.883079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.883168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.883277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.883361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.883479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.883554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.883671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.883744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.883945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.884009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.884399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.884500 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.884583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.884661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.884785 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.884860 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.884973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] publish_errors = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.69/identity {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] publish_errors = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.885994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] standard_threads = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.886979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] standard_threads = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] use_journal = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] use_json = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] use_stderr = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] use_syslog = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] use_journal = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] use_json = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.887900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.887975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] use_stderr = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.888047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.888123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] use_syslog = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.888204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.888282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 20 19:08:52.888353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.888428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.888503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.888631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.888715 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.888792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.888921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.889031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.889156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.889284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.889404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.889581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.889664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.889967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.890114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.890239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.890382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.890516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.890650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.890752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.890905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.890983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.891182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.891341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.891423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.891541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.891653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.891767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.891959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.892076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.892195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.892309 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.892422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.892534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.892656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.892769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.892889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.893001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.893113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.893230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.893350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.893462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.893628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.893741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.893854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.893978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.894102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.894216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.894328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.894446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.894566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.894678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.894801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.894916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.895028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.895145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.895264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.895384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.895497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.895608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.895802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.895916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.896051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.896283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.896399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.896511 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.896754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.896869 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.896981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.897204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.897325 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.897438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.897670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.897790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.897910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.898133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.898252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.898370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.898591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.898706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.898817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.899040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.899165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.899284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.899506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.899620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.899733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.899852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.899965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.900076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.900188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.900306 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.900409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.900543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.900653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.900803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.900900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.901147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.901251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.901358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.901455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.901597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.901696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.901791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.901923 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.902054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.902175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.902290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.902411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.902551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.902669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.902791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.902897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.903008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.903121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.903247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.903363 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.903485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.903601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.903715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.903848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.903963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.904080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.904184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.904307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.904424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.904521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.904628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.904741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.904854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.904974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.905089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.905207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.905311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.906102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.906272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.906463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.906651 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.906651 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.907027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.907151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.907267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.907381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.907505 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.907611 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.907723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.907836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.907949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.908071 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.908186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.908299 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.908299 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.908510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.908624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.908754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.908869 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.908982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.909095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.909207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.909320 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.909441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.909570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.909684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.909781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.909895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.910009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.910130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.910244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.910396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.910628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.910762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.910878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.911070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.911187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.911301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.911415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.911528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.911641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.911763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.911886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.912036 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.912155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.912269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.912269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.912490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.912490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.912712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.912833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.912948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.913062 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.913177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.913298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.913424 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.external = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.913598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.913714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.913829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.913942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.914056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.914183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.914299 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.914413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.password = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.914526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.914640 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] auth.token = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.914763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.914878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.914992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.915104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.915216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.915328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.915448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.915592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.915715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.915837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.915953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.916066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.916180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.916294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.916416 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.916541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.916657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.external = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.916778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.916893 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.917007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.917129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.917245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.917364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.917479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.917621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.password = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.917773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.918005 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] auth.token = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.918134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.918251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.918372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.918488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.918602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.918725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.918851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.918967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.919082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.919208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.919324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.919458 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.919588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.919729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.919846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.919967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.920083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.920205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.920322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.920436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.920550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.920664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.920778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.920901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.921016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.921162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.921883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.922057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.922279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.922465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.922591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.922708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.922941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.923065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.923181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.923306 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.923422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.923536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.923536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.923750 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.923864 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.923987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.924103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.924217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.924332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.924445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.924586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.924710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.924826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.924940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.925055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.925169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.925283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.925406 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.925540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.925540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.925773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.925888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.926013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.926208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.926327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.926434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.926548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.926663 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.926786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.926901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.927015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.927130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.927244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.927358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.927529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.927681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.927913 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.928036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.928152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.928267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.928390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.928508 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.928622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.928737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.928851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.928966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.929116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.929235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.929350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.929464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.929603 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.929717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.929829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.929946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.930061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.930175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.930288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.930288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.930510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.930626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.930740 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.930838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.930951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.931064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.931210 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.931329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.931443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.931557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.931672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.931787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.931909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.932056 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.932182 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.932408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.932528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.932644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.932767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.932884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.932998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.933112 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.933233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.933348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.933470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.933636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.933752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.933866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.933980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.934094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.934220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.934337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.934451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.934565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.934678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.934793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.934916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.935032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.935146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.935260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.935373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.935495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.935618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.935733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.935855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.935969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.936083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.936196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.936318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.936433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.936547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.936681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.936803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.936990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.937198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.937388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.937548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.937666 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.937782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.937898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.938027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.938145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.938259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.938373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.938487 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.938601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.938725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.938841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.938956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.939070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.939189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.939312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.939428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.939542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.939655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.939753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.939867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.939989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.940130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.940249 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.940363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.940477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.940591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.940714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.940829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.940943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.941058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.941172 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.941287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.941440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.941729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.941729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.941952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.942067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.942182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.942310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.942428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.942550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.942665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.942665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.942879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.943000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.943116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.943251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.943408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.943527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.943642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.943764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.943880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.943994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.944341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.944459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.944573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.944695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.944811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.944925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.945039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.945152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.945267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.945388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.945503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.945637 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.945751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.945866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.945979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.946106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.946229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.946338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.946451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.946564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.946677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.946799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.946914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.947027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.947141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.947255 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.947370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.947490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.947605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.947719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.947832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.947945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.948058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.948179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.948293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.948408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.948521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.948633 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] role.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.948730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.948852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] role.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.948967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.949331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.949454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.949612 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.949729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.950001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.950124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.950240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.950360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.950477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.950592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.950741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.950898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.951014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.951127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.951239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.951353 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.951474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.951590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.951703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.951843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.951960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.952074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] role.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.952197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.952332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] role.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.952459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.952637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.952813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.953020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.953020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.953293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.953408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.953582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.953700 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.953828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.953946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.954060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.954174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.954287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.954402 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.954523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.954639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.954754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.954867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.955015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.955129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.955251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.955366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.955480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.955738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.955851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.955973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.955973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.956188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.956301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.956415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.956528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.956649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.956767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.956880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.956993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.957106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.957220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.957342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.957458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.957590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.957696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.957810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.957924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.958050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.958166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.958280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.958393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.958506 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.958628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.958744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.958858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.958971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.959088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.959203 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.959324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.959440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.959553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.959666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.959780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.959892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.960002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.960117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.960360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.960487 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.960604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.960719 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.960841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.960959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.961074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.961189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.961290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.961422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.961600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.961720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.961836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.961972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.962091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.connection = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.962215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.962339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.962455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.962578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.962694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.962807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.962905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.963019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.963133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.963256 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.963371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.963485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.963598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.963598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.963811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.963811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.964056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.964194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.964379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.964568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.964568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.964908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.965030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.965148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.965263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.965406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.connection = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.965663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.965799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.965925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80210) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 19:08:52.966840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.966840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.967778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.967778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.967778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.967778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.967778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 20 19:08:52.967778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80211) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 20 19:08:52.996936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:52.997236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:52.997377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:52.999222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:52.999456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:52.999741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.006141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.008232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.035020 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.035250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.037034 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.037315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.181957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.182985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.234292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.234624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.234889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.235135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.303037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.303238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.303321 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.303398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.303496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.303575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.303684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.303771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.303909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.304419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.304509 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.304586 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.331105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.331340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.331694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.331996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.332376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.332578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.332732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.333064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.333253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.333653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.333989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.333989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.333989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.333989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.344937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.345219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.358425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.358682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.358777 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.358906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.358993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.359126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.359260 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.359395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.359485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.359559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.359622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.359752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80210) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.359849 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.359969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80211) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 20 19:08:53.377500 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.377712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:08:53.377792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.377969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:08:53.378068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.378263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:08:53.378386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.378531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.378644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 20 19:08:53.378790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.378912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.379052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.379178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.379296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.379549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.379961 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.391223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.391502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.391770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.392052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.399400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.399887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.400334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.400461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.400793 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.403657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.403940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.404222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.404617 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.404901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.405163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.406037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.406331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.406605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.406880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.407149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.410976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.414305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.414586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.414869 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.415276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.415565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.415830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.416104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.416380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.416654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.416930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.417199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.434059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.435638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.435942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.441865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.442145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.442424 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.444802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.446619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.446912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.449702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.449985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.452198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.453093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.453375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.455181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.455462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.455740 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.456017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.456290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.456567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.456829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.457098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.460833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.461116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.465776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.466065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.466349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.466635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.466914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.467191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.467463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.467734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.476383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.476671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.481346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.485167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.485454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.487244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.487538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.490482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.492163 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.494092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.495820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.496114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.497854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.498149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.498441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.498728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.499009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.500108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.502116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.505751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.509554 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.509841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.510134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.510421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.510707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.510988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.519815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.520191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.520464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.520737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.528702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.530988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.531519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.531753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.532180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.532454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.532729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.535014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.535298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.535578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.535859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.536143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.540766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.542515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.542789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.545875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.546783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.547069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.547348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.547635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.550783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.551066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.551346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.551618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.551894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.552272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.552453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.561179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.562418 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.562648 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.562948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.563230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.563518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.563801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.573282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.576745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.580851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.582487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.583997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.584244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.584530 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.584821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.587329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.590689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.592234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.592347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.593930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.594040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.594310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.594599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.597130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.597422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.597983 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.598264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.602418 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.604069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.607329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.607595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.607692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.607888 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.607987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.608222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.608338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.616160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80210) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.616439 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.616718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.616994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.617267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.617566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.617709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.617850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.618002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.618127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.618294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.618464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.618662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.618932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.619192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.619464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.619739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80210) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.626806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80211) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 20 19:08:53.627086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.627370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.627649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.627929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.628210 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.628489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.628770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.629043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.629315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.629620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.629887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.630171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.630443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80211) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 20 19:08:53.659010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5653205ae9c0 pid: 80210 (default app) Jul 20 19:08:53.662451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395ba50-6803-4139-97ab-f8d0c7d909a9 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:08:53.662745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395ba50-6803-4139-97ab-f8d0c7d909a9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:08:53.663023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395ba50-6803-4139-97ab-f8d0c7d909a9 None None] PATH_INFO: `/v3` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:08:53.667249 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1/1] 10.222.0.69 () {50 vars in 791 bytes} [Wed Jul 20 19:08:53 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:08:53.669465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5653205ae9c0 pid: 80211 (default app) Jul 20 19:09:01.774399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eeafb-42f6-421c-89ad-dca54f0a4864 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:01.774976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eeafb-42f6-421c-89ad-dca54f0a4864 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:01.774976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1eeafb-42f6-421c-89ad-dca54f0a4864 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:01.776284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 2/2] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:01.784657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:01.785135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:01.785234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:01.860034 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: INFO dbcounter [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] Registered counter for database keystone Jul 20 19:09:01.883796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG oslo_db.sqlalchemy.engines [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80211) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 19:09:01.891020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writer thread running {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 19:09:02.084140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG passlib.handlers.bcrypt [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80211) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 19:09:02.084473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG passlib.handlers.bcrypt [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80211) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 19:09:02.143145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:02.144761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:02.287057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1/3] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:01 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 506 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 19:09:02.488758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eeafb-42f6-421c-89ad-dca54f0a4864 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:02.558532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: INFO dbcounter [None req-7a1eeafb-42f6-421c-89ad-dca54f0a4864 None None] Registered counter for database keystone Jul 20 19:09:02.579582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG oslo_db.sqlalchemy.engines [None req-7a1eeafb-42f6-421c-89ad-dca54f0a4864 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80210) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 20 19:09:02.586268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writer thread running {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 20 19:09:02.706110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1eeafb-42f6-421c-89ad-dca54f0a4864 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:02.710515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640c6522-0b5d-4c13-8972-ccffccbf1c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:02.711183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c6522-0b5d-4c13-8972-ccffccbf1c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:02.711468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c6522-0b5d-4c13-8972-ccffccbf1c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:02.711748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640c6522-0b5d-4c13-8972-ccffccbf1c90 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:02.720537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c6522-0b5d-4c13-8972-ccffccbf1c90 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:02.773582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WARNING keystone.common.rbac_enforcer.enforcer [None req-640c6522-0b5d-4c13-8972-ccffccbf1c90 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 19:09:02.773900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640c6522-0b5d-4c13-8972-ccffccbf1c90 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:02.785069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 3/4] 10.222.0.69 () {58 vars in 1324 bytes} [Wed Jul 20 19:09:02 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:09:02.791158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:02.809298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2800486a-bd08-4d74-af2c-3ec7c84f874e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:02.814765 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72121898-69ae-4781-9fd8-2e79131448a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:02.815924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72121898-69ae-4781-9fd8-2e79131448a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:02.816209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72121898-69ae-4781-9fd8-2e79131448a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:02.816484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72121898-69ae-4781-9fd8-2e79131448a1 admin admin] PATH_INFO: `/v3/projects/1a14d1da63be4e14a0cb08d341815ffe` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:02.818025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72121898-69ae-4781-9fd8-2e79131448a1 admin admin] RBAC: Authorizing `identity:get_project(project_id=1a14d1da63be4e14a0cb08d341815ffe)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:02.873341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: WARNING keystone.common.rbac_enforcer.enforcer [None req-72121898-69ae-4781-9fd8-2e79131448a1 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 20 19:09:02.873709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72121898-69ae-4781-9fd8-2e79131448a1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:02.880572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 2/5] 10.222.0.69 () {56 vars in 1207 bytes} [Wed Jul 20 19:09:02 2022] GET /identity/v3/projects/1a14d1da63be4e14a0cb08d341815ffe => generated 344 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:05.004977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1725a4-6011-4abf-8767-72e24f142e65 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:05.005694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1725a4-6011-4abf-8767-72e24f142e65 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:05.005694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1725a4-6011-4abf-8767-72e24f142e65 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:05.009726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 4/6] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:05 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:05.016008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d5bb6-f26c-4e92-bea4-12d826b13d0f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:05.016545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d5bb6-f26c-4e92-bea4-12d826b13d0f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:05.016545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412d5bb6-f26c-4e92-bea4-12d826b13d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:05.045334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-412d5bb6-f26c-4e92-bea4-12d826b13d0f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:05.047133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-412d5bb6-f26c-4e92-bea4-12d826b13d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:05.055338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 3/7] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:05 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 20 19:09:05.078989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1725a4-6011-4abf-8767-72e24f142e65 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:05.091992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1725a4-6011-4abf-8767-72e24f142e65 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:05.096569 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16eccf28-2ab7-411c-8d49-1213b43b32ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:05.097215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eccf28-2ab7-411c-8d49-1213b43b32ce admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:05.097503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eccf28-2ab7-411c-8d49-1213b43b32ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:05.097808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16eccf28-2ab7-411c-8d49-1213b43b32ce admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:05.106493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eccf28-2ab7-411c-8d49-1213b43b32ce admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:05.108905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16eccf28-2ab7-411c-8d49-1213b43b32ce admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:05.118460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 5/8] 10.222.0.69 () {58 vars in 1324 bytes} [Wed Jul 20 19:09:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:09:05.124878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d5bb6-f26c-4e92-bea4-12d826b13d0f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:05.134225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412d5bb6-f26c-4e92-bea4-12d826b13d0f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:05.138808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3f2f29-849c-4581-bda6-4d01f4b151e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:05.139710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2f29-849c-4581-bda6-4d01f4b151e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:05.139987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2f29-849c-4581-bda6-4d01f4b151e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:05.140258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3f2f29-849c-4581-bda6-4d01f4b151e5 admin admin] PATH_INFO: `/v3/users/c9cf50bf3f924b6dac930c74893f5fb5` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:05.141691 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f2f29-849c-4581-bda6-4d01f4b151e5 admin admin] RBAC: Authorizing `identity:get_user(user_id=c9cf50bf3f924b6dac930c74893f5fb5)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:05.143545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a3f2f29-849c-4581-bda6-4d01f4b151e5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:05.145145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 4/9] 10.222.0.69 () {56 vars in 1198 bytes} [Wed Jul 20 19:09:05 2022] GET /identity/v3/users/c9cf50bf3f924b6dac930c74893f5fb5 => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:07.270925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e63fa-3b6e-4847-b7fe-a1b195f5d6e0 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:07.271470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e63fa-3b6e-4847-b7fe-a1b195f5d6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:07.271470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e63fa-3b6e-4847-b7fe-a1b195f5d6e0 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:07.272817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 6/10] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:07.278004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef93f1-11bf-457f-8901-fd4b92f4b560 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:07.278420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef93f1-11bf-457f-8901-fd4b92f4b560 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:07.278642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cef93f1-11bf-457f-8901-fd4b92f4b560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:07.296564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2cef93f1-11bf-457f-8901-fd4b92f4b560 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:07.297886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2cef93f1-11bf-457f-8901-fd4b92f4b560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:07.349246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d41aa-5efc-4ec2-834d-ea61a02f5eea None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:07.349617 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d41aa-5efc-4ec2-834d-ea61a02f5eea None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:07.349904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d89d41aa-5efc-4ec2-834d-ea61a02f5eea None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:07.351249 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 7/11] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:07.356314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576133d1-262a-4bb1-8afe-cb784f0077e0 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:07.356597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576133d1-262a-4bb1-8afe-cb784f0077e0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:07.356868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576133d1-262a-4bb1-8afe-cb784f0077e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:07.385269 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 5/12] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:07.398076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG passlib.handlers.bcrypt [None req-576133d1-262a-4bb1-8afe-cb784f0077e0 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80210) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 20 19:09:07.398492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG passlib.handlers.bcrypt [None req-576133d1-262a-4bb1-8afe-cb784f0077e0 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80210) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 20 19:09:07.408788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef93f1-11bf-457f-8901-fd4b92f4b560 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:07.432213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:07.444528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-576133d1-262a-4bb1-8afe-cb784f0077e0 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:07.446157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-576133d1-262a-4bb1-8afe-cb784f0077e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:07.512558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cef93f1-11bf-457f-8901-fd4b92f4b560 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:07.545468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d8b7e-4cac-4a83-9d43-9adc59bd5453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:07.549684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 8/13] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:07.549781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d8b7e-4cac-4a83-9d43-9adc59bd5453 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:07.549781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d8b7e-4cac-4a83-9d43-9adc59bd5453 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:07.549781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3d8b7e-4cac-4a83-9d43-9adc59bd5453 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:07.558661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3d8b7e-4cac-4a83-9d43-9adc59bd5453 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:07.569183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576133d1-262a-4bb1-8afe-cb784f0077e0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-ad3d8b7e-4cac-4a83-9d43-9adc59bd5453 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:07.571008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:07.571877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:07.572753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 6/14] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:07.578116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d8b7e-4cac-4a83-9d43-9adc59bd5453 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:07.662133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:07.670407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576133d1-262a-4bb1-8afe-cb784f0077e0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:07.673458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:07.681178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3d8b7e-4cac-4a83-9d43-9adc59bd5453 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:07.715894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749273a0-5d7c-4296-9013-b8280dc92cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:07.716792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749273a0-5d7c-4296-9013-b8280dc92cdb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:07.717079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749273a0-5d7c-4296-9013-b8280dc92cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:07.717353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749273a0-5d7c-4296-9013-b8280dc92cdb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:07.718280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820ed18-87ff-4175-b705-1a669c352a4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:07.719162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820ed18-87ff-4175-b705-1a669c352a4a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:07.719482 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820ed18-87ff-4175-b705-1a669c352a4a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:07.719801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a820ed18-87ff-4175-b705-1a669c352a4a None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:07.720587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820ed18-87ff-4175-b705-1a669c352a4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:07.720919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820ed18-87ff-4175-b705-1a669c352a4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:07.722393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a820ed18-87ff-4175-b705-1a669c352a4a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:07.735435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 7/15] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:07.740959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820ed18-87ff-4175-b705-1a669c352a4a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:07.787691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749273a0-5d7c-4296-9013-b8280dc92cdb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:07.789214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-749273a0-5d7c-4296-9013-b8280dc92cdb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:07.839762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a820ed18-87ff-4175-b705-1a669c352a4a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:07.874316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63992d8-ce5d-441f-9498-454657c4c49f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:07.875016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63992d8-ce5d-441f-9498-454657c4c49f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:07.875300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63992d8-ce5d-441f-9498-454657c4c49f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:07.875575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63992d8-ce5d-441f-9498-454657c4c49f None admin] PATH_INFO: `/v3/users/c9cf50bf3f924b6dac930c74893f5fb5` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:07.876784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63992d8-ce5d-441f-9498-454657c4c49f None admin] RBAC: Authorizing `identity:get_user(user_id=c9cf50bf3f924b6dac930c74893f5fb5)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:07.878288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63992d8-ce5d-441f-9498-454657c4c49f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:07.880099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 8/16] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:07 2022] GET /identity/v3/users/c9cf50bf3f924b6dac930c74893f5fb5 => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:07.885208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63992d8-ce5d-441f-9498-454657c4c49f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:07.890397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 9/17] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:09:07.896183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749273a0-5d7c-4296-9013-b8280dc92cdb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:07.898947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:07.899607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:07.985284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63992d8-ce5d-441f-9498-454657c4c49f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:07.995513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749273a0-5d7c-4296-9013-b8280dc92cdb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:08.020103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c562b2-07ef-4a31-a99f-a9fa359e6bb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:08.020901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c562b2-07ef-4a31-a99f-a9fa359e6bb0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:08.021195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c562b2-07ef-4a31-a99f-a9fa359e6bb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:08.021473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c562b2-07ef-4a31-a99f-a9fa359e6bb0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:08.022881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c562b2-07ef-4a31-a99f-a9fa359e6bb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:08.024343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c562b2-07ef-4a31-a99f-a9fa359e6bb0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:08.025984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 9/18] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:07 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:08.031554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93e1b4-2abd-4b12-baa1-d091610670ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:08.032295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e1b4-2abd-4b12-baa1-d091610670ae None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:08.032392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c562b2-07ef-4a31-a99f-a9fa359e6bb0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:08.032582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e1b4-2abd-4b12-baa1-d091610670ae None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:08.032862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee93e1b4-2abd-4b12-baa1-d091610670ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:08.051877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93e1b4-2abd-4b12-baa1-d091610670ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:08.053350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee93e1b4-2abd-4b12-baa1-d091610670ae None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ee93e1b4-2abd-4b12-baa1-d091610670ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:09:08.073579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:08.074435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:08.075233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:09:08.075233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:09:08.075233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:09:08.075233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:08.075233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 10/19] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:08.081141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93e1b4-2abd-4b12-baa1-d091610670ae None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:08.114538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:08.134583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c562b2-07ef-4a31-a99f-a9fa359e6bb0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:08.139296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:08.171496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d682e5b-2f44-4d85-aba1-639f933b5a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:08.172163 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682e5b-2f44-4d85-aba1-639f933b5a6a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:08.172449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682e5b-2f44-4d85-aba1-639f933b5a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:08.172741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d682e5b-2f44-4d85-aba1-639f933b5a6a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:08.173615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d682e5b-2f44-4d85-aba1-639f933b5a6a None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.domain.id=default, user.id=c9cf50bf3f924b6dac930c74893f5fb5) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:08.173928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d682e5b-2f44-4d85-aba1-639f933b5a6a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:08.175375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d682e5b-2f44-4d85-aba1-639f933b5a6a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:08.182372 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 10/20] 10.222.0.69 () {56 vars in 1322 bytes} [Wed Jul 20 19:09:08 2022] GET /identity/v3/role_assignments?user.id=c9cf50bf3f924b6dac930c74893f5fb5&scope.domain.id=default&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 233 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:08.185630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee93e1b4-2abd-4b12-baa1-d091610670ae None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:08.225110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04de292b-7cd4-425c-a5f7-b57954aa2805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:08.225816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de292b-7cd4-425c-a5f7-b57954aa2805 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:08.226108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de292b-7cd4-425c-a5f7-b57954aa2805 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:08.226390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04de292b-7cd4-425c-a5f7-b57954aa2805 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:08.227142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de292b-7cd4-425c-a5f7-b57954aa2805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:08.227455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de292b-7cd4-425c-a5f7-b57954aa2805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:08.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04de292b-7cd4-425c-a5f7-b57954aa2805 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:08.250259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 11/21] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:09:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:10.561986 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415f1d0-90bd-4779-acab-57d8a534d8c9 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:10.562594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415f1d0-90bd-4779-acab-57d8a534d8c9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:10.562594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4415f1d0-90bd-4779-acab-57d8a534d8c9 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:10.563946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 11/22] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:10.579594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849db5-a1ec-452d-9c98-dc67731ff97b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:10.580080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849db5-a1ec-452d-9c98-dc67731ff97b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:10.580080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74849db5-a1ec-452d-9c98-dc67731ff97b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:10.603552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-74849db5-a1ec-452d-9c98-dc67731ff97b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:10.605113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-74849db5-a1ec-452d-9c98-dc67731ff97b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:10.657621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9931933-2ea2-4125-aff3-32597599df88 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:10.658269 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9931933-2ea2-4125-aff3-32597599df88 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:10.658761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9931933-2ea2-4125-aff3-32597599df88 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:10.660337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 12/23] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:10.665922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24e9e6-5487-414e-87be-a7758214a254 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:10.666466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24e9e6-5487-414e-87be-a7758214a254 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:10.666944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e24e9e6-5487-414e-87be-a7758214a254 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:10.690819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4e24e9e6-5487-414e-87be-a7758214a254 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:10.694001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4e24e9e6-5487-414e-87be-a7758214a254 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:10.728142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 12/24] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:10.729950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd883f-9dae-4d37-8ab0-fb214edd6a57 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:10.730474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd883f-9dae-4d37-8ab0-fb214edd6a57 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:10.730971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84cd883f-9dae-4d37-8ab0-fb214edd6a57 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:10.732487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 13/25] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:10.742808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9c19e-6151-4f00-9d00-5f0ca54995f8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:10.743358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9c19e-6151-4f00-9d00-5f0ca54995f8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:10.743843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87f9c19e-6151-4f00-9d00-5f0ca54995f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:10.775642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-87f9c19e-6151-4f00-9d00-5f0ca54995f8 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:10.778392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-87f9c19e-6151-4f00-9d00-5f0ca54995f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:10.827302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 13/26] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:10.828585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24e9e6-5487-414e-87be-a7758214a254 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:10.862809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:10.911157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 14/27] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:10.912467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9c19e-6151-4f00-9d00-5f0ca54995f8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:10.984654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:11.032325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87f9c19e-6151-4f00-9d00-5f0ca54995f8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:11.063726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e24e9e6-5487-414e-87be-a7758214a254 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:11.068563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc932ba-5b98-4666-b19e-b8398b351b51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:11.069638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc932ba-5b98-4666-b19e-b8398b351b51 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:11.070147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc932ba-5b98-4666-b19e-b8398b351b51 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:11.070679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bc932ba-5b98-4666-b19e-b8398b351b51 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:11.082799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc932ba-5b98-4666-b19e-b8398b351b51 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:11.084734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bc932ba-5b98-4666-b19e-b8398b351b51 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6bc932ba-5b98-4666-b19e-b8398b351b51 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:09:11.104305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:11.105181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:11.106044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:09:11.106044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:09:11.106044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:09:11.106044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:11.108199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 15/28] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:11.109495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc932ba-5b98-4666-b19e-b8398b351b51 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:11.147023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e831b2a-af7f-4796-afbb-82fa149bf26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:11.147666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e831b2a-af7f-4796-afbb-82fa149bf26d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:11.147950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e831b2a-af7f-4796-afbb-82fa149bf26d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:11.148227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e831b2a-af7f-4796-afbb-82fa149bf26d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:11.160828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:11.177676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e831b2a-af7f-4796-afbb-82fa149bf26d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-2e831b2a-af7f-4796-afbb-82fa149bf26d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:11.197910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:11.198873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:11.199656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 14/29] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:11.200317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e831b2a-af7f-4796-afbb-82fa149bf26d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:11.231173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bc932ba-5b98-4666-b19e-b8398b351b51 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:11.268935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce898124-ccd5-41b3-aaab-120690b182d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:11.270016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce898124-ccd5-41b3-aaab-120690b182d2 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:11.270526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce898124-ccd5-41b3-aaab-120690b182d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:11.271010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce898124-ccd5-41b3-aaab-120690b182d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:11.271983 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce898124-ccd5-41b3-aaab-120690b182d2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:11.273624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce898124-ccd5-41b3-aaab-120690b182d2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:11.408373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 16/30] 10.222.0.69 () {60 vars in 1131 bytes} [Wed Jul 20 19:09:11 2022] POST /identity/v3/roles => generated 219 bytes in 299 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:11.409595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce898124-ccd5-41b3-aaab-120690b182d2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:11.486347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e831b2a-af7f-4796-afbb-82fa149bf26d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:11.513927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:11.534390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce898124-ccd5-41b3-aaab-120690b182d2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:11.578045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25cbae9-09ee-474f-9c14-1b01e3824c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:11.578750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cbae9-09ee-474f-9c14-1b01e3824c90 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:11.579041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cbae9-09ee-474f-9c14-1b01e3824c90 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:11.579322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25cbae9-09ee-474f-9c14-1b01e3824c90 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:11.580096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cbae9-09ee-474f-9c14-1b01e3824c90 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:11.580407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cbae9-09ee-474f-9c14-1b01e3824c90 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:11.581955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25cbae9-09ee-474f-9c14-1b01e3824c90 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:11.584469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:11.605931 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 15/31] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:09:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 406 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:11.608491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045dfd1-d5a4-4d50-9b25-544dc1e6e5b2 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:11.608945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045dfd1-d5a4-4d50-9b25-544dc1e6e5b2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:11.609541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3045dfd1-d5a4-4d50-9b25-544dc1e6e5b2 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:11.611489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 16/32] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:11 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:11.612476 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045dfd1-d5a4-4d50-9b25-544dc1e6e5b2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:11.630950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be24c855-8011-411e-8471-1d616cf1e4ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:11.631653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24c855-8011-411e-8471-1d616cf1e4ac None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:11.631942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24c855-8011-411e-8471-1d616cf1e4ac None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:11.632219 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be24c855-8011-411e-8471-1d616cf1e4ac None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:11.632980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24c855-8011-411e-8471-1d616cf1e4ac None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:11.633286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24c855-8011-411e-8471-1d616cf1e4ac None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:11.634869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be24c855-8011-411e-8471-1d616cf1e4ac None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:11.652472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 17/33] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:11.657586 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3411b0b-c8bf-4f8a-9e83-afb96d5549b2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:11.660130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3411b0b-c8bf-4f8a-9e83-afb96d5549b2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:11.660461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3411b0b-c8bf-4f8a-9e83-afb96d5549b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:11.681398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d3411b0b-c8bf-4f8a-9e83-afb96d5549b2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:11.683358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d3411b0b-c8bf-4f8a-9e83-afb96d5549b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:11.754822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3045dfd1-d5a4-4d50-9b25-544dc1e6e5b2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:11.785365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 18/34] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:11.787046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3411b0b-c8bf-4f8a-9e83-afb96d5549b2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:11.810336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead49e1e-8cd5-4e0b-8e87-bf45b32b9280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:11.811006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead49e1e-8cd5-4e0b-8e87-bf45b32b9280 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:11.812493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead49e1e-8cd5-4e0b-8e87-bf45b32b9280 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:11.812853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead49e1e-8cd5-4e0b-8e87-bf45b32b9280 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:11.813943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead49e1e-8cd5-4e0b-8e87-bf45b32b9280 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:11.815641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead49e1e-8cd5-4e0b-8e87-bf45b32b9280 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:11.902770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:11.925293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3411b0b-c8bf-4f8a-9e83-afb96d5549b2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:11.965041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc8c1-6ded-4463-8fd3-be4c7fa44512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:11.965954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc8c1-6ded-4463-8fd3-be4c7fa44512 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:11.966326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc8c1-6ded-4463-8fd3-be4c7fa44512 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:11.966674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ddc8c1-6ded-4463-8fd3-be4c7fa44512 None admin] PATH_INFO: `/v3/users/c9cf50bf3f924b6dac930c74893f5fb5` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:11.969449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddc8c1-6ded-4463-8fd3-be4c7fa44512 None admin] RBAC: Authorizing `identity:get_user(user_id=c9cf50bf3f924b6dac930c74893f5fb5)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:11.971105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ddc8c1-6ded-4463-8fd3-be4c7fa44512 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:11.973557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 19/35] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:11 2022] GET /identity/v3/users/c9cf50bf3f924b6dac930c74893f5fb5 => generated 247 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:11.973910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc8c1-6ded-4463-8fd3-be4c7fa44512 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:11.993710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 17/36] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:09:11 2022] POST /identity/v3/projects => generated 301 bytes in 376 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:11.994959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead49e1e-8cd5-4e0b-8e87-bf45b32b9280 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:12.080879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:12.104642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ddc8c1-6ded-4463-8fd3-be4c7fa44512 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:12.161674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03907110-fc28-49aa-9146-825aa056db8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:12.162368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907110-fc28-49aa-9146-825aa056db8d None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:12.162653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907110-fc28-49aa-9146-825aa056db8d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:12.162939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03907110-fc28-49aa-9146-825aa056db8d None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:12.163717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03907110-fc28-49aa-9146-825aa056db8d None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:12.165152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03907110-fc28-49aa-9146-825aa056db8d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:12.182653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead49e1e-8cd5-4e0b-8e87-bf45b32b9280 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:12.197058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 20/37] 10.222.0.69 () {60 vars in 1131 bytes} [Wed Jul 20 19:09:11 2022] POST /identity/v3/roles => generated 225 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:12.198540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e97fe-2074-4aea-b7bd-1e0e3456dc1f None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:12.198832 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e97fe-2074-4aea-b7bd-1e0e3456dc1f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:12.199114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b23e97fe-2074-4aea-b7bd-1e0e3456dc1f None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:12.200445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 21/38] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:12.216066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933df2f9-53e4-43ef-8140-937813fc1e36 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:12.216395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933df2f9-53e4-43ef-8140-937813fc1e36 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:12.216743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-933df2f9-53e4-43ef-8140-937813fc1e36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:12.250621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-933df2f9-53e4-43ef-8140-937813fc1e36 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:12.252147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-933df2f9-53e4-43ef-8140-937813fc1e36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:12.254600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:12.257430 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc2779-0080-4029-b5fc-eb28aea13aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:12.258754 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc2779-0080-4029-b5fc-eb28aea13aa3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:12.259284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc2779-0080-4029-b5fc-eb28aea13aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:12.259765 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccc2779-0080-4029-b5fc-eb28aea13aa3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:12.266559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc2779-0080-4029-b5fc-eb28aea13aa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:12.269618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ccc2779-0080-4029-b5fc-eb28aea13aa3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:12.271519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 18/39] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:12.278665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc2779-0080-4029-b5fc-eb28aea13aa3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:12.368839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 22/40] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:12.370643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fed9ac-c849-4195-9d9c-ddadb4046a8b None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:12.370643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fed9ac-c849-4195-9d9c-ddadb4046a8b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:12.370949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fed9ac-c849-4195-9d9c-ddadb4046a8b None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:12.372408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 23/41] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:12.378262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec812d44-e7c4-4efa-8fd2-fcb23ca83b58 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:12.378579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec812d44-e7c4-4efa-8fd2-fcb23ca83b58 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:12.378836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec812d44-e7c4-4efa-8fd2-fcb23ca83b58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:12.399578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ec812d44-e7c4-4efa-8fd2-fcb23ca83b58 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:12.400967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ec812d44-e7c4-4efa-8fd2-fcb23ca83b58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:12.422133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccc2779-0080-4029-b5fc-eb28aea13aa3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:12.436935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:12.459407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f533ff0-8b9f-4bc7-a19c-5b59cd70a7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:12.460289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f533ff0-8b9f-4bc7-a19c-5b59cd70a7e2 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:12.460635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f533ff0-8b9f-4bc7-a19c-5b59cd70a7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:12.460928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f533ff0-8b9f-4bc7-a19c-5b59cd70a7e2 None admin] PATH_INFO: `/v3/domains/default/users/c9cf50bf3f924b6dac930c74893f5fb5/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:12.463098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f533ff0-8b9f-4bc7-a19c-5b59cd70a7e2 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=c9cf50bf3f924b6dac930c74893f5fb5, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:12.464551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f533ff0-8b9f-4bc7-a19c-5b59cd70a7e2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:12.475568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 24/42] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:12.476419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec812d44-e7c4-4efa-8fd2-fcb23ca83b58 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:12.485613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 19/43] 10.222.0.69 () {58 vars in 1361 bytes} [Wed Jul 20 19:09:12 2022] PUT /identity/v3/domains/default/users/c9cf50bf3f924b6dac930c74893f5fb5/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:12.500666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f533ff0-8b9f-4bc7-a19c-5b59cd70a7e2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:12.560248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:12.602225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec812d44-e7c4-4efa-8fd2-fcb23ca83b58 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:12.619106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f533ff0-8b9f-4bc7-a19c-5b59cd70a7e2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:12.663185 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb6aee-5834-4b27-8efc-998832825731 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:12.663880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb6aee-5834-4b27-8efc-998832825731 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:12.664165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb6aee-5834-4b27-8efc-998832825731 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:12.664443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cb6aee-5834-4b27-8efc-998832825731 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:12.665201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb6aee-5834-4b27-8efc-998832825731 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:12.666678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cb6aee-5834-4b27-8efc-998832825731 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:12.684724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 20/44] 10.222.0.69 () {60 vars in 1131 bytes} [Wed Jul 20 19:09:12 2022] POST /identity/v3/roles => generated 223 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:09:12.775795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c31a0ebc-7fb1-4f2d-9c8a-1ff99c543eae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:12.776900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a0ebc-7fb1-4f2d-9c8a-1ff99c543eae None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:12.777303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a0ebc-7fb1-4f2d-9c8a-1ff99c543eae None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:12.777717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c31a0ebc-7fb1-4f2d-9c8a-1ff99c543eae None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:12.779491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a0ebc-7fb1-4f2d-9c8a-1ff99c543eae None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:12.783052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c31a0ebc-7fb1-4f2d-9c8a-1ff99c543eae None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:12.785807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 25/45] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:12.792018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb6aee-5834-4b27-8efc-998832825731 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:12.832558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:13.011204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cb6aee-5834-4b27-8efc-998832825731 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:13.044472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034f0afe-65b7-41b4-86a9-7f1c73d49c3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:13.045060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f0afe-65b7-41b4-86a9-7f1c73d49c3f None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:13.045347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f0afe-65b7-41b4-86a9-7f1c73d49c3f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:13.045909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034f0afe-65b7-41b4-86a9-7f1c73d49c3f None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:13.046639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f0afe-65b7-41b4-86a9-7f1c73d49c3f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:13.048105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034f0afe-65b7-41b4-86a9-7f1c73d49c3f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:13.073294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 21/46] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:09:12 2022] POST /identity/v3/projects => generated 312 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:14.563137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0214fb4-982c-4df2-bb7c-5bfaad996c56 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:14.563649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0214fb4-982c-4df2-bb7c-5bfaad996c56 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:14.563759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0214fb4-982c-4df2-bb7c-5bfaad996c56 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:14.565087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 26/47] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:14.570661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfee8d2-eff4-490e-86bd-732e18881625 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:14.571056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfee8d2-eff4-490e-86bd-732e18881625 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:14.571293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfee8d2-eff4-490e-86bd-732e18881625 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:14.589350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-bbfee8d2-eff4-490e-86bd-732e18881625 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:14.590920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bbfee8d2-eff4-490e-86bd-732e18881625 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:14.659316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 22/48] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:14.682444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0214fb4-982c-4df2-bb7c-5bfaad996c56 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:14.787390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0214fb4-982c-4df2-bb7c-5bfaad996c56 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:14.825223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35902a27-61af-45aa-89ab-730218a1ccea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:14.825965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35902a27-61af-45aa-89ab-730218a1ccea None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:14.826253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35902a27-61af-45aa-89ab-730218a1ccea None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:14.826530 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35902a27-61af-45aa-89ab-730218a1ccea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:14.831175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:14.839745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35902a27-61af-45aa-89ab-730218a1ccea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-35902a27-61af-45aa-89ab-730218a1ccea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:14.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:14.849987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:14.850980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 27/49] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:14.857662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfee8d2-eff4-490e-86bd-732e18881625 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:14.876495 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:14.960410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfee8d2-eff4-490e-86bd-732e18881625 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:14.994295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9379c6-58d1-489c-93cf-3d107ba5a94f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:14.994978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9379c6-58d1-489c-93cf-3d107ba5a94f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:14.995266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9379c6-58d1-489c-93cf-3d107ba5a94f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:14.995544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9379c6-58d1-489c-93cf-3d107ba5a94f None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:14.996293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9379c6-58d1-489c-93cf-3d107ba5a94f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:14.996602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9379c6-58d1-489c-93cf-3d107ba5a94f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:14.998029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd9379c6-58d1-489c-93cf-3d107ba5a94f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:15.007511 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 23/50] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:15.013019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35902a27-61af-45aa-89ab-730218a1ccea None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:15.113493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35902a27-61af-45aa-89ab-730218a1ccea None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:15.134486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848552eb-58ba-4826-80f9-c746258a305e None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:15.134781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848552eb-58ba-4826-80f9-c746258a305e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:15.135065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848552eb-58ba-4826-80f9-c746258a305e None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:15.136464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 24/51] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:15.141485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425146d-ed48-4043-9c06-ce7834e4d3d6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:15.141808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425146d-ed48-4043-9c06-ce7834e4d3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:15.142091 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4425146d-ed48-4043-9c06-ce7834e4d3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:15.151512 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc55f7-624a-4398-9eac-10b9f10c4230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:15.152388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc55f7-624a-4398-9eac-10b9f10c4230 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:15.152679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc55f7-624a-4398-9eac-10b9f10c4230 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:15.152962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bc55f7-624a-4398-9eac-10b9f10c4230 None admin] PATH_INFO: `/v3/users/c9cf50bf3f924b6dac930c74893f5fb5` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:15.154186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc55f7-624a-4398-9eac-10b9f10c4230 None admin] RBAC: Authorizing `identity:get_user(user_id=c9cf50bf3f924b6dac930c74893f5fb5)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:15.155862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bc55f7-624a-4398-9eac-10b9f10c4230 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:15.157514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 28/52] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:15 2022] GET /identity/v3/users/c9cf50bf3f924b6dac930c74893f5fb5 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:15.158901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4425146d-ed48-4043-9c06-ce7834e4d3d6 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:15.160106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4425146d-ed48-4043-9c06-ce7834e4d3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:15.162746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc55f7-624a-4398-9eac-10b9f10c4230 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:15.227297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:15.228717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 25/53] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:15.254339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425146d-ed48-4043-9c06-ce7834e4d3d6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:15.264480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:15.270060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bc55f7-624a-4398-9eac-10b9f10c4230 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:15.305734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0f01-8c84-4b93-af63-d2d262675e49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:15.306528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0f01-8c84-4b93-af63-d2d262675e49 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:15.306814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0f01-8c84-4b93-af63-d2d262675e49 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:15.307096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23be0f01-8c84-4b93-af63-d2d262675e49 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:15.308277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be0f01-8c84-4b93-af63-d2d262675e49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:15.310208 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23be0f01-8c84-4b93-af63-d2d262675e49 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:15.312088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 29/54] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:15.318381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0f01-8c84-4b93-af63-d2d262675e49 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:15.361374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4425146d-ed48-4043-9c06-ce7834e4d3d6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:15.396997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f202cf-d602-45db-a98a-049e69c41cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:15.397762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f202cf-d602-45db-a98a-049e69c41cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:15.398052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f202cf-d602-45db-a98a-049e69c41cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:15.398332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f202cf-d602-45db-a98a-049e69c41cd8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:15.399514 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f202cf-d602-45db-a98a-049e69c41cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:15.400985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f202cf-d602-45db-a98a-049e69c41cd8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:15.402926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 26/55] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:15.409280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f202cf-d602-45db-a98a-049e69c41cd8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:15.420003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23be0f01-8c84-4b93-af63-d2d262675e49 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:15.448297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:15.454943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69e6322-942d-472b-bd8a-ffa2d14ba620 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:15.455616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e6322-942d-472b-bd8a-ffa2d14ba620 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:15.456005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e6322-942d-472b-bd8a-ffa2d14ba620 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:15.456304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69e6322-942d-472b-bd8a-ffa2d14ba620 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:15.457141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e6322-942d-472b-bd8a-ffa2d14ba620 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.domain.id=default, user.id=c9cf50bf3f924b6dac930c74893f5fb5) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:15.457461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e6322-942d-472b-bd8a-ffa2d14ba620 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:15.458959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69e6322-942d-472b-bd8a-ffa2d14ba620 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:15.467747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 30/56] 10.222.0.69 () {56 vars in 1322 bytes} [Wed Jul 20 19:09:15 2022] GET /identity/v3/role_assignments?user.id=c9cf50bf3f924b6dac930c74893f5fb5&scope.domain.id=default&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 530 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:15.488722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:15.513055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f202cf-d602-45db-a98a-049e69c41cd8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:15.550444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-710a1f62-400e-4889-9767-55cff3899180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:15.551135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a1f62-400e-4889-9767-55cff3899180 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:15.551425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a1f62-400e-4889-9767-55cff3899180 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:15.551708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-710a1f62-400e-4889-9767-55cff3899180 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:15.552434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a1f62-400e-4889-9767-55cff3899180 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:15.553960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-710a1f62-400e-4889-9767-55cff3899180 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:15.581624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 27/57] 10.222.0.69 () {60 vars in 1140 bytes} [Wed Jul 20 19:09:15 2022] POST /identity/v3/projects => generated 298 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:09:17.644329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf6d8-13d9-4bfc-af21-ff0f7d231e34 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:17.644845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf6d8-13d9-4bfc-af21-ff0f7d231e34 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:17.644845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbdcf6d8-13d9-4bfc-af21-ff0f7d231e34 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:17.646358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 31/58] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:17.651835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206150a-3a6e-481a-9b6b-838725d224ff None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:17.652257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206150a-3a6e-481a-9b6b-838725d224ff None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:17.652403 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e206150a-3a6e-481a-9b6b-838725d224ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:17.669953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e206150a-3a6e-481a-9b6b-838725d224ff None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:17.671478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e206150a-3a6e-481a-9b6b-838725d224ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:17.738571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 28/59] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:17.763961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf6d8-13d9-4bfc-af21-ff0f7d231e34 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:17.874730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbdcf6d8-13d9-4bfc-af21-ff0f7d231e34 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:17.908149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12951558-47b0-4f68-acbb-703c48462de4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:17.908938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12951558-47b0-4f68-acbb-703c48462de4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:17.909220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12951558-47b0-4f68-acbb-703c48462de4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:17.909496 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12951558-47b0-4f68-acbb-703c48462de4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:17.910718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12951558-47b0-4f68-acbb-703c48462de4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:17.912181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12951558-47b0-4f68-acbb-703c48462de4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:17.913847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 32/60] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:17.919584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206150a-3a6e-481a-9b6b-838725d224ff None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:17.981864 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:18.024955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e206150a-3a6e-481a-9b6b-838725d224ff None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:18.058732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71809dd-3cc1-4a92-81f1-1241a0b0181b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:18.059414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71809dd-3cc1-4a92-81f1-1241a0b0181b None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:18.059701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71809dd-3cc1-4a92-81f1-1241a0b0181b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:18.059979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71809dd-3cc1-4a92-81f1-1241a0b0181b None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:18.060698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71809dd-3cc1-4a92-81f1-1241a0b0181b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:18.063010 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71809dd-3cc1-4a92-81f1-1241a0b0181b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:18.088833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 29/61] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:09:17 2022] POST /identity/v3/users => generated 275 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:21.368516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c1eb7-46ab-46f0-b9b4-bf797dcdda95 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:21.369467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c1eb7-46ab-46f0-b9b4-bf797dcdda95 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:21.369988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f0c1eb7-46ab-46f0-b9b4-bf797dcdda95 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:21.371556 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 33/62] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:21.378017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206acf4-6509-4cb6-8f31-873eb41cf2b1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:21.378944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206acf4-6509-4cb6-8f31-873eb41cf2b1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:21.379421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1206acf4-6509-4cb6-8f31-873eb41cf2b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:21.405733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1206acf4-6509-4cb6-8f31-873eb41cf2b1 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:21.407257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1206acf4-6509-4cb6-8f31-873eb41cf2b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:21.489286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e1ab3-b033-4de7-8739-2157ba0397f8 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:21.489929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e1ab3-b033-4de7-8739-2157ba0397f8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:21.490401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8e1ab3-b033-4de7-8739-2157ba0397f8 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:21.491960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 34/63] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:21.499339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cad182-27ca-4b7d-bf04-f09156cd49b7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:21.499902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cad182-27ca-4b7d-bf04-f09156cd49b7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:21.500413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cad182-27ca-4b7d-bf04-f09156cd49b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:21.526783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b1cad182-27ca-4b7d-bf04-f09156cd49b7 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:21.528643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b1cad182-27ca-4b7d-bf04-f09156cd49b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:21.533002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 30/64] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:21.560562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206acf4-6509-4cb6-8f31-873eb41cf2b1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:21.615253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:21.641882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 35/65] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 19:09:21.667818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:21.699599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cad182-27ca-4b7d-bf04-f09156cd49b7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:21.745909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1206acf4-6509-4cb6-8f31-873eb41cf2b1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:21.795420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1036a-b1d5-4e7c-8b06-9428fc8bea3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:21.796242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1036a-b1d5-4e7c-8b06-9428fc8bea3b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:21.796532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1036a-b1d5-4e7c-8b06-9428fc8bea3b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:21.796813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1036a-b1d5-4e7c-8b06-9428fc8bea3b None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:21.810688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1036a-b1d5-4e7c-8b06-9428fc8bea3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a4c1036a-b1d5-4e7c-8b06-9428fc8bea3b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:21.821456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:21.822370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:21.823145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 31/66] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:21.824564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e2f0e-5e8b-423b-bb57-3262c1dd27de None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:21.824862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e2f0e-5e8b-423b-bb57-3262c1dd27de None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:21.825145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7e2f0e-5e8b-423b-bb57-3262c1dd27de None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:21.826480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 32/67] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:21.832871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e2f0e-5e8b-423b-bb57-3262c1dd27de None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:21.847225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cad182-27ca-4b7d-bf04-f09156cd49b7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:21.897468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7cd9640-f545-42d4-80f0-21782adb8917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:21.898262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd9640-f545-42d4-80f0-21782adb8917 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:21.898550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd9640-f545-42d4-80f0-21782adb8917 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:21.898830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7cd9640-f545-42d4-80f0-21782adb8917 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:21.910300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7cd9640-f545-42d4-80f0-21782adb8917 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b7cd9640-f545-42d4-80f0-21782adb8917 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:21.917959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:21.918821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:21.919597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 36/68] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:21.920442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7042b-31a6-4e7e-8829-db18fdb103ae None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:21.920897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7042b-31a6-4e7e-8829-db18fdb103ae None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:21.921325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e7042b-31a6-4e7e-8829-db18fdb103ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:21.939690 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-18e7042b-31a6-4e7e-8829-db18fdb103ae None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:21.940911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-18e7042b-31a6-4e7e-8829-db18fdb103ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:21.967032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7e2f0e-5e8b-423b-bb57-3262c1dd27de None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:21.970710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:21.993853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:22.009707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b372bd-382d-47c8-a68e-9a8d720ca39d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:22.010389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b372bd-382d-47c8-a68e-9a8d720ca39d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.010684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b372bd-382d-47c8-a68e-9a8d720ca39d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.010965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b372bd-382d-47c8-a68e-9a8d720ca39d None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.011722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b372bd-382d-47c8-a68e-9a8d720ca39d None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:22.012039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b372bd-382d-47c8-a68e-9a8d720ca39d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:22.013445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b372bd-382d-47c8-a68e-9a8d720ca39d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:22.021759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 37/69] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:22.021759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b680f9-a8b9-45df-98bc-5f24f07c596b None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.021759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b680f9-a8b9-45df-98bc-5f24f07c596b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.021759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b680f9-a8b9-45df-98bc-5f24f07c596b None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.021759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 38/70] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:22.021759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b680f9-a8b9-45df-98bc-5f24f07c596b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:22.031380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 33/71] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:22.033231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd42a0-7004-4d65-9eca-1c3007861a08 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.033760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd42a0-7004-4d65-9eca-1c3007861a08 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.034248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febd42a0-7004-4d65-9eca-1c3007861a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.054885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-febd42a0-7004-4d65-9eca-1c3007861a08 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:22.056389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-febd42a0-7004-4d65-9eca-1c3007861a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:22.128025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 34/72] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:22.128768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd42a0-7004-4d65-9eca-1c3007861a08 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:22.142211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b680f9-a8b9-45df-98bc-5f24f07c596b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:22.180958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ebb69-b94f-45b0-b94c-044e02d0b80e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:22.181575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ebb69-b94f-45b0-b94c-044e02d0b80e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.181869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ebb69-b94f-45b0-b94c-044e02d0b80e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.182150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7ebb69-b94f-45b0-b94c-044e02d0b80e None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.182909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ebb69-b94f-45b0-b94c-044e02d0b80e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:22.183220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ebb69-b94f-45b0-b94c-044e02d0b80e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:22.184645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7ebb69-b94f-45b0-b94c-044e02d0b80e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:22.196750 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 39/73] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:22.196750 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ebb69-b94f-45b0-b94c-044e02d0b80e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:22.227872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:22.235468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febd42a0-7004-4d65-9eca-1c3007861a08 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:22.253417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:22.271862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2803f8c-98e4-40de-a2fc-4804e95358ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:22.272716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2803f8c-98e4-40de-a2fc-4804e95358ad None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.272996 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2803f8c-98e4-40de-a2fc-4804e95358ad None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.273266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2803f8c-98e4-40de-a2fc-4804e95358ad None admin] PATH_INFO: `/v3/users/343f2a00f3ed43acb00bd614cb69766e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.287772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2803f8c-98e4-40de-a2fc-4804e95358ad None admin] RBAC: Authorizing `identity:get_user(user_id=343f2a00f3ed43acb00bd614cb69766e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:22.289233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2803f8c-98e4-40de-a2fc-4804e95358ad None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:22.290965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 35/74] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:22 2022] GET /identity/v3/users/343f2a00f3ed43acb00bd614cb69766e => generated 275 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:22.292256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c000d8-6d2f-45e2-958f-e2794d602df1 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.292549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c000d8-6d2f-45e2-958f-e2794d602df1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.292941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47c000d8-6d2f-45e2-958f-e2794d602df1 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.294401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 36/75] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:22.300090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c000d8-6d2f-45e2-958f-e2794d602df1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:22.316225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7ebb69-b94f-45b0-b94c-044e02d0b80e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:22.353291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee27562-0fef-461d-b9ac-401ca9bb5772 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:22.354120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee27562-0fef-461d-b9ac-401ca9bb5772 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.354411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee27562-0fef-461d-b9ac-401ca9bb5772 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.354697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee27562-0fef-461d-b9ac-401ca9bb5772 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.365272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee27562-0fef-461d-b9ac-401ca9bb5772 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2ee27562-0fef-461d-b9ac-401ca9bb5772 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:22.373231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:22.375519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 40/76] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:09:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:22.376279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee27562-0fef-461d-b9ac-401ca9bb5772 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:22.413199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47c000d8-6d2f-45e2-958f-e2794d602df1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:22.451121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786e6c9-fc61-4181-a510-f59bf896e25c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:22.451865 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786e6c9-fc61-4181-a510-f59bf896e25c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.452159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786e6c9-fc61-4181-a510-f59bf896e25c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.452454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c786e6c9-fc61-4181-a510-f59bf896e25c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.463913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c786e6c9-fc61-4181-a510-f59bf896e25c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-c786e6c9-fc61-4181-a510-f59bf896e25c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:22.472197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:22.473345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 37/77] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:22.479028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786e6c9-fc61-4181-a510-f59bf896e25c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:22.487359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee27562-0fef-461d-b9ac-401ca9bb5772 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:22.493951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:22.495241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:22.530660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48d8f32-0fff-4d3a-a1d5-20cc67db4265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:22.531528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8f32-0fff-4d3a-a1d5-20cc67db4265 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.531818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8f32-0fff-4d3a-a1d5-20cc67db4265 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.532098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48d8f32-0fff-4d3a-a1d5-20cc67db4265 None admin] PATH_INFO: `/v3/users/c9cf50bf3f924b6dac930c74893f5fb5` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.533320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d8f32-0fff-4d3a-a1d5-20cc67db4265 None admin] RBAC: Authorizing `identity:get_user(user_id=c9cf50bf3f924b6dac930c74893f5fb5)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:22.534958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48d8f32-0fff-4d3a-a1d5-20cc67db4265 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:22.536619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 41/78] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:22 2022] GET /identity/v3/users/c9cf50bf3f924b6dac930c74893f5fb5 => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:22.537841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f6347-da3f-4cb9-803d-cee2da48f0d1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.538130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f6347-da3f-4cb9-803d-cee2da48f0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.538410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80f6347-da3f-4cb9-803d-cee2da48f0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.555516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d80f6347-da3f-4cb9-803d-cee2da48f0d1 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:22.556748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d80f6347-da3f-4cb9-803d-cee2da48f0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:22.590094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c786e6c9-fc61-4181-a510-f59bf896e25c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:22.626009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 42/79] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:22.626220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4421e1-d62f-4cc8-ba02-e20752566313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:22.626790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4421e1-d62f-4cc8-ba02-e20752566313 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.626895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f6347-da3f-4cb9-803d-cee2da48f0d1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:22.627051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4421e1-d62f-4cc8-ba02-e20752566313 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.627302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4421e1-d62f-4cc8-ba02-e20752566313 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.697335 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4421e1-d62f-4cc8-ba02-e20752566313 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:22.698863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4421e1-d62f-4cc8-ba02-e20752566313 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:22.713984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:22.740446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80f6347-da3f-4cb9-803d-cee2da48f0d1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:22.742767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:22.779380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d383ed-754c-43c2-a8fd-19bb578b30ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:22.780141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d383ed-754c-43c2-a8fd-19bb578b30ec None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.780432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d383ed-754c-43c2-a8fd-19bb578b30ec None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.780717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d383ed-754c-43c2-a8fd-19bb578b30ec None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.781475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d383ed-754c-43c2-a8fd-19bb578b30ec None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:22.781818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d383ed-754c-43c2-a8fd-19bb578b30ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:22.783225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d383ed-754c-43c2-a8fd-19bb578b30ec None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:22.791204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 43/80] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:09:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:22.792007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d383ed-754c-43c2-a8fd-19bb578b30ec None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:22.805063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 38/81] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:22.806073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4421e1-d62f-4cc8-ba02-e20752566313 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:22.899334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d383ed-754c-43c2-a8fd-19bb578b30ec None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:22.922112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4421e1-d62f-4cc8-ba02-e20752566313 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:22.934746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235df5a-169b-4ae5-ac1a-2dc56ba68547 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:22.935800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235df5a-169b-4ae5-ac1a-2dc56ba68547 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.936760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235df5a-169b-4ae5-ac1a-2dc56ba68547 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.937040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:22.937184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1235df5a-169b-4ae5-ac1a-2dc56ba68547 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.938035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235df5a-169b-4ae5-ac1a-2dc56ba68547 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:22.938442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235df5a-169b-4ae5-ac1a-2dc56ba68547 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:22.939922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1235df5a-169b-4ae5-ac1a-2dc56ba68547 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:22.948781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 44/82] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:22.949627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235df5a-169b-4ae5-ac1a-2dc56ba68547 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:22.964737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2572d-c201-4ccb-8331-d5582f4a9884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:22.965321 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2572d-c201-4ccb-8331-d5582f4a9884 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:22.965663 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2572d-c201-4ccb-8331-d5582f4a9884 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:22.965948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9e2572d-c201-4ccb-8331-d5582f4a9884 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:22.980352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:23.040051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2572d-c201-4ccb-8331-d5582f4a9884 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:23.041580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9e2572d-c201-4ccb-8331-d5582f4a9884 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:23.057759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1235df5a-169b-4ae5-ac1a-2dc56ba68547 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:23.092379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958ca3b-2701-4dda-99fe-5741c75c6bd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:23.093338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958ca3b-2701-4dda-99fe-5741c75c6bd2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:23.093663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958ca3b-2701-4dda-99fe-5741c75c6bd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:23.093950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6958ca3b-2701-4dda-99fe-5741c75c6bd2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:23.095174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958ca3b-2701-4dda-99fe-5741c75c6bd2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:23.096656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6958ca3b-2701-4dda-99fe-5741c75c6bd2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:23.098607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 45/83] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:22 2022] GET /identity/v3/domains/default => generated 199 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:23.099376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958ca3b-2701-4dda-99fe-5741c75c6bd2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:23.152021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 39/84] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:23.152885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2572d-c201-4ccb-8331-d5582f4a9884 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:23.169292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:23.197187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:23.203825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6958ca3b-2701-4dda-99fe-5741c75c6bd2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:23.239530 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b7a08-2f32-4829-b7a1-a7741f85c853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:23.240327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7a08-2f32-4829-b7a1-a7741f85c853 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:23.240620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7a08-2f32-4829-b7a1-a7741f85c853 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:23.240909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459b7a08-2f32-4829-b7a1-a7741f85c853 None admin] PATH_INFO: `/v3/users/343f2a00f3ed43acb00bd614cb69766e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:23.242158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b7a08-2f32-4829-b7a1-a7741f85c853 None admin] RBAC: Authorizing `identity:get_user(user_id=343f2a00f3ed43acb00bd614cb69766e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:23.243606 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459b7a08-2f32-4829-b7a1-a7741f85c853 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:23.245252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 46/85] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:23 2022] GET /identity/v3/users/343f2a00f3ed43acb00bd614cb69766e => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:23.246046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b7a08-2f32-4829-b7a1-a7741f85c853 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:23.263613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9e2572d-c201-4ccb-8331-d5582f4a9884 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:23.298434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5438b5-9285-4298-8dc7-a2ae46fb374d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:23.299319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5438b5-9285-4298-8dc7-a2ae46fb374d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:23.299601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5438b5-9285-4298-8dc7-a2ae46fb374d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:23.299881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5438b5-9285-4298-8dc7-a2ae46fb374d None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:23.301085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5438b5-9285-4298-8dc7-a2ae46fb374d None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:23.302572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5438b5-9285-4298-8dc7-a2ae46fb374d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:23.304297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 40/86] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:23 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:23.305041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5438b5-9285-4298-8dc7-a2ae46fb374d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:23.358426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459b7a08-2f32-4829-b7a1-a7741f85c853 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:23.392111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:23.396517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a772f-7b8c-442b-a10e-838cafa4340a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:23.397404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a772f-7b8c-442b-a10e-838cafa4340a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:23.397820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a772f-7b8c-442b-a10e-838cafa4340a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:23.398108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4a772f-7b8c-442b-a10e-838cafa4340a None admin] PATH_INFO: `/v3/users/343f2a00f3ed43acb00bd614cb69766e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:23.399335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a772f-7b8c-442b-a10e-838cafa4340a None admin] RBAC: Authorizing `identity:get_user(user_id=343f2a00f3ed43acb00bd614cb69766e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:23.400778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4a772f-7b8c-442b-a10e-838cafa4340a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:23.402456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 47/87] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:23 2022] GET /identity/v3/users/343f2a00f3ed43acb00bd614cb69766e => generated 275 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:23.403221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a772f-7b8c-442b-a10e-838cafa4340a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:23.414504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5438b5-9285-4298-8dc7-a2ae46fb374d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:23.439206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:23.449664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cda746-f30b-4e02-9e92-4269f51d1d03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:23.450483 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cda746-f30b-4e02-9e92-4269f51d1d03 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:23.450786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cda746-f30b-4e02-9e92-4269f51d1d03 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:23.451133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20cda746-f30b-4e02-9e92-4269f51d1d03 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:23.452374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cda746-f30b-4e02-9e92-4269f51d1d03 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:23.453862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20cda746-f30b-4e02-9e92-4269f51d1d03 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:23.474624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 41/88] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:09:23 2022] POST /identity/v3/users => generated 289 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:23.475532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cda746-f30b-4e02-9e92-4269f51d1d03 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:23.528704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4a772f-7b8c-442b-a10e-838cafa4340a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:23.575950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b421b-0ec4-4fb5-93c5-7a0507dfd20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:23.576998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b421b-0ec4-4fb5-93c5-7a0507dfd20c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:23.577398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b421b-0ec4-4fb5-93c5-7a0507dfd20c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:23.577722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-150b421b-0ec4-4fb5-93c5-7a0507dfd20c None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:23.579202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b421b-0ec4-4fb5-93c5-7a0507dfd20c None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:23.581647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-150b421b-0ec4-4fb5-93c5-7a0507dfd20c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:23.583546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 48/89] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:23 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:23.584362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b421b-0ec4-4fb5-93c5-7a0507dfd20c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:23.615082 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20cda746-f30b-4e02-9e92-4269f51d1d03 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:23.659920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886ecf3-f696-4ac8-b8bb-ffa814a59235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:23.660871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886ecf3-f696-4ac8-b8bb-ffa814a59235 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:23.661168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886ecf3-f696-4ac8-b8bb-ffa814a59235 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:23.661451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f886ecf3-f696-4ac8-b8bb-ffa814a59235 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:23.675102 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:23.748905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:23.755121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-150b421b-0ec4-4fb5-93c5-7a0507dfd20c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:23.760539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886ecf3-f696-4ac8-b8bb-ffa814a59235 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:23.762143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f886ecf3-f696-4ac8-b8bb-ffa814a59235 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:23.814762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7bb7a-d7a3-4498-b3e8-fab6631ee347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:23.815346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7bb7a-d7a3-4498-b3e8-fab6631ee347 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:23.815635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7bb7a-d7a3-4498-b3e8-fab6631ee347 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:23.815915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b7bb7a-d7a3-4498-b3e8-fab6631ee347 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:23.816753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7bb7a-d7a3-4498-b3e8-fab6631ee347 None admin] RBAC: Adding query filter params (role.id=91f9d5c6716547c18da989bf327f6aca, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4, user.id=343f2a00f3ed43acb00bd614cb69766e) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:23.817123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7bb7a-d7a3-4498-b3e8-fab6631ee347 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:23.819123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b7bb7a-d7a3-4498-b3e8-fab6631ee347 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:23.826268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 49/90] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:23 2022] GET /identity/v3/role_assignments?user.id=343f2a00f3ed43acb00bd614cb69766e&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=91f9d5c6716547c18da989bf327f6aca => generated 259 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:23.828771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7bb7a-d7a3-4498-b3e8-fab6631ee347 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:23.938552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 42/91] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:23.939756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886ecf3-f696-4ac8-b8bb-ffa814a59235 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:23.992224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:24.079128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f886ecf3-f696-4ac8-b8bb-ffa814a59235 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:24.104113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b7bb7a-d7a3-4498-b3e8-fab6631ee347 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:24.143049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058d3ca-5e30-4c8c-b5fe-56f8cdb75bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:24.143814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058d3ca-5e30-4c8c-b5fe-56f8cdb75bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:24.144103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058d3ca-5e30-4c8c-b5fe-56f8cdb75bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:24.144382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058d3ca-5e30-4c8c-b5fe-56f8cdb75bc5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:24.167265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f34bef-71bb-430f-8767-7f9f8f85d568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:24.167952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f34bef-71bb-430f-8767-7f9f8f85d568 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:24.168245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f34bef-71bb-430f-8767-7f9f8f85d568 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:24.168534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f34bef-71bb-430f-8767-7f9f8f85d568 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:24.169391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f34bef-71bb-430f-8767-7f9f8f85d568 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4, user.id=c9cf50bf3f924b6dac930c74893f5fb5) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:24.169743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f34bef-71bb-430f-8767-7f9f8f85d568 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:24.171403 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f34bef-71bb-430f-8767-7f9f8f85d568 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:24.180163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:24.188345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 43/92] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:23 2022] GET /identity/v3/role_assignments?user.id=c9cf50bf3f924b6dac930c74893f5fb5&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:24.189298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f34bef-71bb-430f-8767-7f9f8f85d568 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:24.227403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058d3ca-5e30-4c8c-b5fe-56f8cdb75bc5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:24.229318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058d3ca-5e30-4c8c-b5fe-56f8cdb75bc5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:24.376005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 50/93] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 549 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:24.384633 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058d3ca-5e30-4c8c-b5fe-56f8cdb75bc5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:24.479806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:24.495513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f34bef-71bb-430f-8767-7f9f8f85d568 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:24.497413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:24.522458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058d3ca-5e30-4c8c-b5fe-56f8cdb75bc5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:24.559900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4248701-99ee-46dd-aa6a-a2b23e700736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:24.561755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4248701-99ee-46dd-aa6a-a2b23e700736 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:24.562393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4248701-99ee-46dd-aa6a-a2b23e700736 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:24.562865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4248701-99ee-46dd-aa6a-a2b23e700736 None admin] PATH_INFO: `/v3/projects/13ffb9be5ed64a9bb86d078f371e308b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:24.565019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4248701-99ee-46dd-aa6a-a2b23e700736 None admin] RBAC: Authorizing `identity:get_project(project_id=13ffb9be5ed64a9bb86d078f371e308b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:24.567839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4248701-99ee-46dd-aa6a-a2b23e700736 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:24.570616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 51/94] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:24 2022] GET /identity/v3/projects/13ffb9be5ed64a9bb86d078f371e308b => generated 312 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:24.580360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4248701-99ee-46dd-aa6a-a2b23e700736 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:24.590882 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc5aed-446b-455e-b18a-7c7d9ab7f24a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:24.591854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc5aed-446b-455e-b18a-7c7d9ab7f24a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:24.592183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc5aed-446b-455e-b18a-7c7d9ab7f24a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:24.592470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fc5aed-446b-455e-b18a-7c7d9ab7f24a None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:24.598604 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc5aed-446b-455e-b18a-7c7d9ab7f24a None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:24.600109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fc5aed-446b-455e-b18a-7c7d9ab7f24a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:24.602184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 44/95] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:24 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 413 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:24.610503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc5aed-446b-455e-b18a-7c7d9ab7f24a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:24.716774 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4248701-99ee-46dd-aa6a-a2b23e700736 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:24.760039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40212822-3585-4a49-9eff-2a007ae72080 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:24.760798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40212822-3585-4a49-9eff-2a007ae72080 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:24.761236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40212822-3585-4a49-9eff-2a007ae72080 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:24.761626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40212822-3585-4a49-9eff-2a007ae72080 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:24.762646 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40212822-3585-4a49-9eff-2a007ae72080 None admin] RBAC: Adding query filter params (role.id=91f9d5c6716547c18da989bf327f6aca, scope.project.id=13ffb9be5ed64a9bb86d078f371e308b, user.id=343f2a00f3ed43acb00bd614cb69766e) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:24.763031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40212822-3585-4a49-9eff-2a007ae72080 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:24.764604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40212822-3585-4a49-9eff-2a007ae72080 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:24.771843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 52/96] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:24 2022] GET /identity/v3/role_assignments?user.id=343f2a00f3ed43acb00bd614cb69766e&scope.project.id=13ffb9be5ed64a9bb86d078f371e308b&role.id=91f9d5c6716547c18da989bf327f6aca => generated 259 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:24.773085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:24.790630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fc5aed-446b-455e-b18a-7c7d9ab7f24a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:24.846454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f45f0-4de3-4d4a-ac64-ba6a71e3889d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:24.847160 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f45f0-4de3-4d4a-ac64-ba6a71e3889d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:24.847451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f45f0-4de3-4d4a-ac64-ba6a71e3889d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:24.847738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f45f0-4de3-4d4a-ac64-ba6a71e3889d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:24.848873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f45f0-4de3-4d4a-ac64-ba6a71e3889d None admin] RBAC: Adding query filter params (role.id=77052d58af4f4b5d9b2d003531462e50, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4, user.id=343f2a00f3ed43acb00bd614cb69766e) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:24.849199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f45f0-4de3-4d4a-ac64-ba6a71e3889d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:24.850989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f45f0-4de3-4d4a-ac64-ba6a71e3889d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:24.857826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 45/97] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:24 2022] GET /identity/v3/role_assignments?user.id=343f2a00f3ed43acb00bd614cb69766e&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=77052d58af4f4b5d9b2d003531462e50 => generated 259 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:26.521608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c05bce-3d63-4f43-a8f1-4e7a942e1942 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:26.522627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c05bce-3d63-4f43-a8f1-4e7a942e1942 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:26.523141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44c05bce-3d63-4f43-a8f1-4e7a942e1942 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:26.524728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 53/98] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:26.542201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12563354-cd99-4ecc-99e4-e0b236482f8f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:26.543121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12563354-cd99-4ecc-99e4-e0b236482f8f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:26.543595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12563354-cd99-4ecc-99e4-e0b236482f8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:26.570845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-12563354-cd99-4ecc-99e4-e0b236482f8f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:26.572382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-12563354-cd99-4ecc-99e4-e0b236482f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:26.615537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:26.683520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 46/99] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:26.723848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c05bce-3d63-4f43-a8f1-4e7a942e1942 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:26.882333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44c05bce-3d63-4f43-a8f1-4e7a942e1942 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:26.927790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b535d0-c5f4-4052-a2a7-4fc06179c09a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:26.928657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b535d0-c5f4-4052-a2a7-4fc06179c09a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:26.928954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b535d0-c5f4-4052-a2a7-4fc06179c09a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:26.929240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b535d0-c5f4-4052-a2a7-4fc06179c09a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:26.930496 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b535d0-c5f4-4052-a2a7-4fc06179c09a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:26.931960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b535d0-c5f4-4052-a2a7-4fc06179c09a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:26.934305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 54/100] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:26.940929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12563354-cd99-4ecc-99e4-e0b236482f8f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:27.071871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12563354-cd99-4ecc-99e4-e0b236482f8f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:27.114199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc2e0de-101c-47ca-9349-778448012a26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:27.115115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2e0de-101c-47ca-9349-778448012a26 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.115415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2e0de-101c-47ca-9349-778448012a26 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.115702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc2e0de-101c-47ca-9349-778448012a26 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.116443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2e0de-101c-47ca-9349-778448012a26 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:27.117966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc2e0de-101c-47ca-9349-778448012a26 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:27.156569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 47/101] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:09:26 2022] POST /identity/v3/projects => generated 302 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:27.173877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619bede-8ec6-4969-974b-5ab963f48bf3 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.174177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619bede-8ec6-4969-974b-5ab963f48bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.174445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619bede-8ec6-4969-974b-5ab963f48bf3 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.175928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 55/102] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:27.193407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684a5e-a7c0-40cf-9b73-94e61ecc42c1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.193748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684a5e-a7c0-40cf-9b73-94e61ecc42c1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.194087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07684a5e-a7c0-40cf-9b73-94e61ecc42c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.214378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647ca95-a614-44be-bbc6-8a15337d5d08 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.214679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647ca95-a614-44be-bbc6-8a15337d5d08 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.214950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f647ca95-a614-44be-bbc6-8a15337d5d08 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.216446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 56/103] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:27.220001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-07684a5e-a7c0-40cf-9b73-94e61ecc42c1 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:27.222582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ed190-fc5d-449c-8d7a-2fe9f8f2e8bf None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.222877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ed190-fc5d-449c-8d7a-2fe9f8f2e8bf None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.223440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282ed190-fc5d-449c-8d7a-2fe9f8f2e8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.228722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-07684a5e-a7c0-40cf-9b73-94e61ecc42c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:27.254315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-282ed190-fc5d-449c-8d7a-2fe9f8f2e8bf None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:27.256354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-282ed190-fc5d-449c-8d7a-2fe9f8f2e8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:27.264991 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:27.370580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:27.383013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 57/105] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:27.383209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 48/105] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:27.416764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684a5e-a7c0-40cf-9b73-94e61ecc42c1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:27.428656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ed190-fc5d-449c-8d7a-2fe9f8f2e8bf None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:27.542483 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07684a5e-a7c0-40cf-9b73-94e61ecc42c1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:27.589713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be216c7d-6b07-47ef-94d3-e55b5d220fe0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:27.590535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be216c7d-6b07-47ef-94d3-e55b5d220fe0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.590827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be216c7d-6b07-47ef-94d3-e55b5d220fe0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.591109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be216c7d-6b07-47ef-94d3-e55b5d220fe0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.598840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282ed190-fc5d-449c-8d7a-2fe9f8f2e8bf None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:27.604419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be216c7d-6b07-47ef-94d3-e55b5d220fe0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-be216c7d-6b07-47ef-94d3-e55b5d220fe0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:27.612868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:27.613803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:27.614583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 49/106] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:27.615753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6daa6-4451-4a40-a62a-ac9df8786577 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.616044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6daa6-4451-4a40-a62a-ac9df8786577 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.616327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d6daa6-4451-4a40-a62a-ac9df8786577 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.621731 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 50/107] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:27.628252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4cca4-ddc4-4b0e-aa8d-0dd6e5354889 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.628408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4cca4-ddc4-4b0e-aa8d-0dd6e5354889 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.628703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa4cca4-ddc4-4b0e-aa8d-0dd6e5354889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.641888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:27.647348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff43e08-6843-455c-aac6-25da03594859 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:27.648486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff43e08-6843-455c-aac6-25da03594859 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.648769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-eaa4cca4-ddc4-4b0e-aa8d-0dd6e5354889 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:27.649211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff43e08-6843-455c-aac6-25da03594859 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.649901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff43e08-6843-455c-aac6-25da03594859 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.650006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eaa4cca4-ddc4-4b0e-aa8d-0dd6e5354889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:27.663919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff43e08-6843-455c-aac6-25da03594859 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6ff43e08-6843-455c-aac6-25da03594859 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:27.681047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:27.682823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:27.684485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:27.689887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 58/108] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:27 2022] GET /identity/v3/roles/reader => generated 84 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:27.691931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff43e08-6843-455c-aac6-25da03594859 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:27.722818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 51/109] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 19:09:27.723724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4cca4-ddc4-4b0e-aa8d-0dd6e5354889 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:27.781650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:27.843489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff43e08-6843-455c-aac6-25da03594859 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:27.850249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa4cca4-ddc4-4b0e-aa8d-0dd6e5354889 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:27.886645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404cb36b-c957-4ffe-890f-a5f054478544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:27.887422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cb36b-c957-4ffe-890f-a5f054478544 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.887722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cb36b-c957-4ffe-890f-a5f054478544 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.888010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404cb36b-c957-4ffe-890f-a5f054478544 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.889488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cb36b-c957-4ffe-890f-a5f054478544 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:27.889945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cb36b-c957-4ffe-890f-a5f054478544 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:27.891707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-404cb36b-c957-4ffe-890f-a5f054478544 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:27.905365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64641a2-9a67-4356-a3e6-0b0efdb10aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:27.906147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64641a2-9a67-4356-a3e6-0b0efdb10aee None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.906451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64641a2-9a67-4356-a3e6-0b0efdb10aee None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.906727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64641a2-9a67-4356-a3e6-0b0efdb10aee None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.909450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 59/110] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:27.915104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f4593-c45d-4c81-82a2-725de2fe54e1 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:27.915104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f4593-c45d-4c81-82a2-725de2fe54e1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:27.915104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-464f4593-c45d-4c81-82a2-725de2fe54e1 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:27.915104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 60/111] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:27.915104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f4593-c45d-4c81-82a2-725de2fe54e1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:27.916004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64641a2-9a67-4356-a3e6-0b0efdb10aee None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:27.916393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64641a2-9a67-4356-a3e6-0b0efdb10aee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:27.918411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64641a2-9a67-4356-a3e6-0b0efdb10aee None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:27.921103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:27.931214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 52/112] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:27 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:27.932356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64641a2-9a67-4356-a3e6-0b0efdb10aee None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:28.050906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-464f4593-c45d-4c81-82a2-725de2fe54e1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:28.065360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64641a2-9a67-4356-a3e6-0b0efdb10aee None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:28.069942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:28.093974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eb4481-d091-4422-85cb-7a358880e1cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:28.094870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb4481-d091-4422-85cb-7a358880e1cc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.095177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb4481-d091-4422-85cb-7a358880e1cc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.095635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eb4481-d091-4422-85cb-7a358880e1cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.106262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eb4481-d091-4422-85cb-7a358880e1cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:28.110157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be87d1-d553-44e9-896f-3af971a85d8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:28.111085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be87d1-d553-44e9-896f-3af971a85d8b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.111378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be87d1-d553-44e9-896f-3af971a85d8b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.111777 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62be87d1-d553-44e9-896f-3af971a85d8b None admin] PATH_INFO: `/v3/users/c9cf50bf3f924b6dac930c74893f5fb5` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.113189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be87d1-d553-44e9-896f-3af971a85d8b None admin] RBAC: Authorizing `identity:get_user(user_id=c9cf50bf3f924b6dac930c74893f5fb5)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:28.114847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62be87d1-d553-44e9-896f-3af971a85d8b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-95eb4481-d091-4422-85cb-7a358880e1cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:28.114959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:28.115787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:28.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 61/113] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:27 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:28.117951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89102bdb-ff97-48ff-a89d-2746bdfd0676 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.118244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89102bdb-ff97-48ff-a89d-2746bdfd0676 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.118517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89102bdb-ff97-48ff-a89d-2746bdfd0676 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.121550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 53/114] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:27 2022] GET /identity/v3/users/c9cf50bf3f924b6dac930c74893f5fb5 => generated 247 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:28.122199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be87d1-d553-44e9-896f-3af971a85d8b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:28.142583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-89102bdb-ff97-48ff-a89d-2746bdfd0676 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:28.143809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-89102bdb-ff97-48ff-a89d-2746bdfd0676 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:28.200703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:28.230073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 62/115] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:28.235850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301efd8-614d-4111-a00c-82fcd7d2602b None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.236582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301efd8-614d-4111-a00c-82fcd7d2602b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.237842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3301efd8-614d-4111-a00c-82fcd7d2602b None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.243262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 63/116] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:28 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:28.244627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301efd8-614d-4111-a00c-82fcd7d2602b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:28.247235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62be87d1-d553-44e9-896f-3af971a85d8b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:28.291613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88a251-5bf8-49e2-bc3f-c188fe2c14da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:28.292507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88a251-5bf8-49e2-bc3f-c188fe2c14da None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.292803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88a251-5bf8-49e2-bc3f-c188fe2c14da None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.293093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c88a251-5bf8-49e2-bc3f-c188fe2c14da None admin] PATH_INFO: `/v3/users/acff0c0a0c4346f697ad5aa1f0334d80` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.309951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88a251-5bf8-49e2-bc3f-c188fe2c14da None admin] RBAC: Authorizing `identity:get_user(user_id=acff0c0a0c4346f697ad5aa1f0334d80)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:28.312370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c88a251-5bf8-49e2-bc3f-c188fe2c14da None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:28.319955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 54/117] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:28 2022] GET /identity/v3/users/acff0c0a0c4346f697ad5aa1f0334d80 => generated 289 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:28.319955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88a251-5bf8-49e2-bc3f-c188fe2c14da None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:28.373288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3301efd8-614d-4111-a00c-82fcd7d2602b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:28.384404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:28.417115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f468c2-d262-4e8e-9278-4d735978dad5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:28.418192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f468c2-d262-4e8e-9278-4d735978dad5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.418488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f468c2-d262-4e8e-9278-4d735978dad5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.418968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f468c2-d262-4e8e-9278-4d735978dad5 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.419687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f468c2-d262-4e8e-9278-4d735978dad5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:28.419995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f468c2-d262-4e8e-9278-4d735978dad5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:28.421440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f468c2-d262-4e8e-9278-4d735978dad5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:28.430321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 64/118] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:28.435160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517344fe-0546-4bb0-b680-125dd5552b7b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.435160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517344fe-0546-4bb0-b680-125dd5552b7b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.435160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517344fe-0546-4bb0-b680-125dd5552b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.445868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c88a251-5bf8-49e2-bc3f-c188fe2c14da None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:28.450660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-517344fe-0546-4bb0-b680-125dd5552b7b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:28.452456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-517344fe-0546-4bb0-b680-125dd5552b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:28.477028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:28.495409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e888d-3ff1-465e-b108-e5dee4342407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:28.496293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e888d-3ff1-465e-b108-e5dee4342407 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.496586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e888d-3ff1-465e-b108-e5dee4342407 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.496946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e888d-3ff1-465e-b108-e5dee4342407 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.498163 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e888d-3ff1-465e-b108-e5dee4342407 None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:28.500132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e888d-3ff1-465e-b108-e5dee4342407 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:28.502114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 55/119] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:28 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:28.507102 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e888d-3ff1-465e-b108-e5dee4342407 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:28.537129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 65/120] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:28.538359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517344fe-0546-4bb0-b680-125dd5552b7b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:28.639912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e888d-3ff1-465e-b108-e5dee4342407 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:28.677188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:28.684354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517344fe-0546-4bb0-b680-125dd5552b7b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:28.699323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb142-b80e-4360-ae2d-0d21a8a8fe2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:28.700148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb142-b80e-4360-ae2d-0d21a8a8fe2a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.700440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb142-b80e-4360-ae2d-0d21a8a8fe2a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.700725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13efb142-b80e-4360-ae2d-0d21a8a8fe2a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.717470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13efb142-b80e-4360-ae2d-0d21a8a8fe2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:28.727760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee3949-9d94-4bfa-8be0-4039ff92704d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:28.728414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee3949-9d94-4bfa-8be0-4039ff92704d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-13efb142-b80e-4360-ae2d-0d21a8a8fe2a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:28.728520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:28.729316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:28.730121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:28.730701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee3949-9d94-4bfa-8be0-4039ff92704d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.730701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aee3949-9d94-4bfa-8be0-4039ff92704d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.731440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 56/121] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:09:28 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:28.731749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb142-b80e-4360-ae2d-0d21a8a8fe2a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:28.795782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:28.814817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee3949-9d94-4bfa-8be0-4039ff92704d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:28.816854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aee3949-9d94-4bfa-8be0-4039ff92704d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:28.854818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13efb142-b80e-4360-ae2d-0d21a8a8fe2a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:28.895941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47581c64-62fe-485e-82ea-bc7581429f55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:28.896838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47581c64-62fe-485e-82ea-bc7581429f55 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:28.897122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47581c64-62fe-485e-82ea-bc7581429f55 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:28.897430 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47581c64-62fe-485e-82ea-bc7581429f55 None admin] PATH_INFO: `/v3/users/343f2a00f3ed43acb00bd614cb69766e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:28.898692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47581c64-62fe-485e-82ea-bc7581429f55 None admin] RBAC: Authorizing `identity:get_user(user_id=343f2a00f3ed43acb00bd614cb69766e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:28.900165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47581c64-62fe-485e-82ea-bc7581429f55 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:28.901888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 57/122] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:28 2022] GET /identity/v3/users/343f2a00f3ed43acb00bd614cb69766e => generated 275 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:28.902640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47581c64-62fe-485e-82ea-bc7581429f55 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:28.933935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:28.945975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 66/123] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:28.946764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee3949-9d94-4bfa-8be0-4039ff92704d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:29.039067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47581c64-62fe-485e-82ea-bc7581429f55 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:29.061429 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:29.066716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aee3949-9d94-4bfa-8be0-4039ff92704d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:29.079380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb73af-cc17-4250-a358-08b94250d21f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:29.080595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb73af-cc17-4250-a358-08b94250d21f None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:29.081045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb73af-cc17-4250-a358-08b94250d21f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:29.081329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb73af-cc17-4250-a358-08b94250d21f None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/users/c9cf50bf3f924b6dac930c74893f5fb5/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:29.083379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb73af-cc17-4250-a358-08b94250d21f None admin] RBAC: Authorizing `identity:create_grant(project_id=a60cff69e3074f9bb2d0f231a1fb2be4, user_id=c9cf50bf3f924b6dac930c74893f5fb5, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:29.084841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9eb73af-cc17-4250-a358-08b94250d21f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:29.102070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 58/124] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:28 2022] PUT /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/users/c9cf50bf3f924b6dac930c74893f5fb5/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:29.102921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb73af-cc17-4250-a358-08b94250d21f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:29.120923 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa637cec-04e5-45b0-93be-c8e87f1295fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:29.122123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa637cec-04e5-45b0-93be-c8e87f1295fd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:29.122369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa637cec-04e5-45b0-93be-c8e87f1295fd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:29.122650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa637cec-04e5-45b0-93be-c8e87f1295fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:29.136897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa637cec-04e5-45b0-93be-c8e87f1295fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-fa637cec-04e5-45b0-93be-c8e87f1295fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:29.146521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:29.147836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 67/125] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:29.149351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa637cec-04e5-45b0-93be-c8e87f1295fd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:29.234258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:29.266096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9eb73af-cc17-4250-a358-08b94250d21f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:29.298462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa637cec-04e5-45b0-93be-c8e87f1295fd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:29.324195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d5cde-8ebe-4f80-bfeb-472027eb5106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:29.324887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d5cde-8ebe-4f80-bfeb-472027eb5106 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:29.325170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d5cde-8ebe-4f80-bfeb-472027eb5106 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:29.325455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d42d5cde-8ebe-4f80-bfeb-472027eb5106 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:29.326248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d5cde-8ebe-4f80-bfeb-472027eb5106 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:29.327194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d5cde-8ebe-4f80-bfeb-472027eb5106 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:29.328766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d42d5cde-8ebe-4f80-bfeb-472027eb5106 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:29.343866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 59/126] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:09:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:29.345070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d5cde-8ebe-4f80-bfeb-472027eb5106 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:29.349945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24297584-9831-4e52-8907-cfe102dbf931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:29.351051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24297584-9831-4e52-8907-cfe102dbf931 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:29.351466 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24297584-9831-4e52-8907-cfe102dbf931 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:29.351951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24297584-9831-4e52-8907-cfe102dbf931 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:29.354005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24297584-9831-4e52-8907-cfe102dbf931 None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:29.355770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24297584-9831-4e52-8907-cfe102dbf931 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:29.357801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 68/127] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:29 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:29.364030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24297584-9831-4e52-8907-cfe102dbf931 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:29.410685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:29.486322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d42d5cde-8ebe-4f80-bfeb-472027eb5106 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:29.493457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24297584-9831-4e52-8907-cfe102dbf931 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:29.522779 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:29.532659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d316444-0125-4b5a-bb9b-eed6ce455b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:29.533451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d316444-0125-4b5a-bb9b-eed6ce455b72 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:29.533764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d316444-0125-4b5a-bb9b-eed6ce455b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:29.534043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d316444-0125-4b5a-bb9b-eed6ce455b72 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:29.534907 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d316444-0125-4b5a-bb9b-eed6ce455b72 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:29.535310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d316444-0125-4b5a-bb9b-eed6ce455b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:29.537009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d316444-0125-4b5a-bb9b-eed6ce455b72 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:29.537607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aca859-e31d-44b4-b08e-e8a8c40cb172 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:29.539022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aca859-e31d-44b4-b08e-e8a8c40cb172 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:29.539992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aca859-e31d-44b4-b08e-e8a8c40cb172 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:29.540481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7aca859-e31d-44b4-b08e-e8a8c40cb172 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:29.541975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aca859-e31d-44b4-b08e-e8a8c40cb172 None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:29.544392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7aca859-e31d-44b4-b08e-e8a8c40cb172 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:29.551817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 60/129] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:29 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:29.551956 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 69/129] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:29.552855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d316444-0125-4b5a-bb9b-eed6ce455b72 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:29.553255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aca859-e31d-44b4-b08e-e8a8c40cb172 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:29.672489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d316444-0125-4b5a-bb9b-eed6ce455b72 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:29.702890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7aca859-e31d-44b4-b08e-e8a8c40cb172 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:29.715204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:29.716920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34676896-b056-472f-8393-26157626bd6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:29.719023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34676896-b056-472f-8393-26157626bd6a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:29.719312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34676896-b056-472f-8393-26157626bd6a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:29.719594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34676896-b056-472f-8393-26157626bd6a None admin] PATH_INFO: `/v3/users/343f2a00f3ed43acb00bd614cb69766e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:29.722533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34676896-b056-472f-8393-26157626bd6a None admin] RBAC: Authorizing `identity:get_user(user_id=343f2a00f3ed43acb00bd614cb69766e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:29.724337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34676896-b056-472f-8393-26157626bd6a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:29.726077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 70/130] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:29 2022] GET /identity/v3/users/343f2a00f3ed43acb00bd614cb69766e => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:29.726831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34676896-b056-472f-8393-26157626bd6a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:29.752689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0f0a3-abc4-4e4d-b625-960d335133ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:29.753613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0f0a3-abc4-4e4d-b625-960d335133ca None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:29.753908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0f0a3-abc4-4e4d-b625-960d335133ca None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:29.754190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b0f0a3-abc4-4e4d-b625-960d335133ca None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/users/343f2a00f3ed43acb00bd614cb69766e/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:29.756183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0f0a3-abc4-4e4d-b625-960d335133ca None admin] RBAC: Authorizing `identity:create_grant(project_id=a60cff69e3074f9bb2d0f231a1fb2be4, user_id=343f2a00f3ed43acb00bd614cb69766e, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:29.757715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b0f0a3-abc4-4e4d-b625-960d335133ca None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:29.777587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 61/131] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:29 2022] PUT /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/users/343f2a00f3ed43acb00bd614cb69766e/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:29.777587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0f0a3-abc4-4e4d-b625-960d335133ca None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:29.815035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:29.881861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34676896-b056-472f-8393-26157626bd6a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:29.929500 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ab0b38-acd0-4534-ac4b-38694cd48bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:29.930263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab0b38-acd0-4534-ac4b-38694cd48bae None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:29.930560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab0b38-acd0-4534-ac4b-38694cd48bae None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:29.930848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ab0b38-acd0-4534-ac4b-38694cd48bae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:29.931721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab0b38-acd0-4534-ac4b-38694cd48bae None admin] RBAC: Adding query filter params (role.id=997417d78f914e27958b852313dcae0e, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4, user.id=acff0c0a0c4346f697ad5aa1f0334d80) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:29.932048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab0b38-acd0-4534-ac4b-38694cd48bae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:29.933804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ab0b38-acd0-4534-ac4b-38694cd48bae None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:29.938178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b0f0a3-abc4-4e4d-b625-960d335133ca None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:29.944889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 71/132] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:29 2022] GET /identity/v3/role_assignments?user.id=acff0c0a0c4346f697ad5aa1f0334d80&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=997417d78f914e27958b852313dcae0e => generated 259 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:29.944889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2349edf-aee5-4898-a221-2bbcc468f663 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:29.944889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2349edf-aee5-4898-a221-2bbcc468f663 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:29.944889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2349edf-aee5-4898-a221-2bbcc468f663 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:29.944889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 72/133] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:29.945323 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2349edf-aee5-4898-a221-2bbcc468f663 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:30.007866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5af710-22ea-447d-b473-72a5e0ee60fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:30.008927 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5af710-22ea-447d-b473-72a5e0ee60fe None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:30.009288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5af710-22ea-447d-b473-72a5e0ee60fe None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:30.009708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5af710-22ea-447d-b473-72a5e0ee60fe None admin] PATH_INFO: `/v3/users/343f2a00f3ed43acb00bd614cb69766e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:30.011917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5af710-22ea-447d-b473-72a5e0ee60fe None admin] RBAC: Authorizing `identity:get_user(user_id=343f2a00f3ed43acb00bd614cb69766e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:30.013541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5af710-22ea-447d-b473-72a5e0ee60fe None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:30.015425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 62/134] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:29 2022] GET /identity/v3/users/343f2a00f3ed43acb00bd614cb69766e => generated 275 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:30.021847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4bca7-3b22-4b49-83cb-ab051dac86ec None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:30.022147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4bca7-3b22-4b49-83cb-ab051dac86ec None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:30.022438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd4bca7-3b22-4b49-83cb-ab051dac86ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:30.042372 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7dd4bca7-3b22-4b49-83cb-ab051dac86ec None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:30.043663 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7dd4bca7-3b22-4b49-83cb-ab051dac86ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:30.082611 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:30.104797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2349edf-aee5-4898-a221-2bbcc468f663 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:30.128271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 63/135] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:30.129250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4bca7-3b22-4b49-83cb-ab051dac86ec None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:30.140474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:30.168002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb2d93-dd29-4cdc-8b07-ee928b00baf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:30.169014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb2d93-dd29-4cdc-8b07-ee928b00baf1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:30.169259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb2d93-dd29-4cdc-8b07-ee928b00baf1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:30.169615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-accb2d93-dd29-4cdc-8b07-ee928b00baf1 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:30.174362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb2d93-dd29-4cdc-8b07-ee928b00baf1 None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:30.175966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-accb2d93-dd29-4cdc-8b07-ee928b00baf1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:30.178266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 73/136] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:29 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:30.184459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb2d93-dd29-4cdc-8b07-ee928b00baf1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:30.240512 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd4bca7-3b22-4b49-83cb-ab051dac86ec None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:30.278394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf211c5-615f-4828-967c-0994b807d4a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:30.279318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf211c5-615f-4828-967c-0994b807d4a3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:30.279660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf211c5-615f-4828-967c-0994b807d4a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:30.279940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf211c5-615f-4828-967c-0994b807d4a3 None admin] PATH_INFO: `/v3/projects/13ffb9be5ed64a9bb86d078f371e308b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:30.281409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf211c5-615f-4828-967c-0994b807d4a3 None admin] RBAC: Authorizing `identity:get_project(project_id=13ffb9be5ed64a9bb86d078f371e308b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:30.282970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf211c5-615f-4828-967c-0994b807d4a3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:30.287613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 64/137] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:30 2022] GET /identity/v3/projects/13ffb9be5ed64a9bb86d078f371e308b => generated 312 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:30.288523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf211c5-615f-4828-967c-0994b807d4a3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:30.316480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-accb2d93-dd29-4cdc-8b07-ee928b00baf1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:30.337646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:30.358447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8f23f-e976-41f1-a66a-9d700f144be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:30.359582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8f23f-e976-41f1-a66a-9d700f144be9 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:30.359901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8f23f-e976-41f1-a66a-9d700f144be9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:30.360231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e8f23f-e976-41f1-a66a-9d700f144be9 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/users/343f2a00f3ed43acb00bd614cb69766e/roles/77052d58af4f4b5d9b2d003531462e50` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:30.362852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8f23f-e976-41f1-a66a-9d700f144be9 None admin] RBAC: Authorizing `identity:create_grant(project_id=a60cff69e3074f9bb2d0f231a1fb2be4, user_id=343f2a00f3ed43acb00bd614cb69766e, role_id=77052d58af4f4b5d9b2d003531462e50)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:30.364528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e8f23f-e976-41f1-a66a-9d700f144be9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:30.385179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 74/138] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:30 2022] PUT /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/users/343f2a00f3ed43acb00bd614cb69766e/roles/77052d58af4f4b5d9b2d003531462e50 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:30.386302 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8f23f-e976-41f1-a66a-9d700f144be9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:30.407357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf211c5-615f-4828-967c-0994b807d4a3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:30.446659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1862b-3c90-4121-8139-a83270f4bfe4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:30.447533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1862b-3c90-4121-8139-a83270f4bfe4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:30.447826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1862b-3c90-4121-8139-a83270f4bfe4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:30.448114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a1862b-3c90-4121-8139-a83270f4bfe4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:30.449459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1862b-3c90-4121-8139-a83270f4bfe4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:30.451085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2a1862b-3c90-4121-8139-a83270f4bfe4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:30.453800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 65/139] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:30.454144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:30.461685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1862b-3c90-4121-8139-a83270f4bfe4 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:30.528237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e8f23f-e976-41f1-a66a-9d700f144be9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:30.572420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90accfa8-dc07-4063-8e36-42b9be4e0354 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:30.573407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90accfa8-dc07-4063-8e36-42b9be4e0354 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:30.573726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90accfa8-dc07-4063-8e36-42b9be4e0354 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:30.574007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90accfa8-dc07-4063-8e36-42b9be4e0354 None admin] PATH_INFO: `/v3/projects/13ffb9be5ed64a9bb86d078f371e308b/users/343f2a00f3ed43acb00bd614cb69766e/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:30.577539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90accfa8-dc07-4063-8e36-42b9be4e0354 None admin] RBAC: Authorizing `identity:create_grant(project_id=13ffb9be5ed64a9bb86d078f371e308b, user_id=343f2a00f3ed43acb00bd614cb69766e, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:30.579459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90accfa8-dc07-4063-8e36-42b9be4e0354 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:30.600882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 75/140] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:30 2022] PUT /identity/v3/projects/13ffb9be5ed64a9bb86d078f371e308b/users/343f2a00f3ed43acb00bd614cb69766e/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:30.616178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a1862b-3c90-4121-8139-a83270f4bfe4 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:30.630746 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:30.671605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26c554dc-1bbf-4597-8cad-4f8c64139564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:30.672530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c554dc-1bbf-4597-8cad-4f8c64139564 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:30.673031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c554dc-1bbf-4597-8cad-4f8c64139564 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:30.673568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26c554dc-1bbf-4597-8cad-4f8c64139564 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:30.674373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c554dc-1bbf-4597-8cad-4f8c64139564 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:30.676031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26c554dc-1bbf-4597-8cad-4f8c64139564 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:30.703745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 66/141] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:09:30 2022] POST /identity/v3/users => generated 283 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:31.611229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deee31f-9594-42fd-9e05-b86ef6741ef4 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:31.611886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deee31f-9594-42fd-9e05-b86ef6741ef4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:31.611886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8deee31f-9594-42fd-9e05-b86ef6741ef4 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:31.613078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 76/142] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:31.619253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fd75f-b404-4057-80bb-0e395c193f73 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:31.619803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fd75f-b404-4057-80bb-0e395c193f73 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:31.620270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-627fd75f-b404-4057-80bb-0e395c193f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:31.645313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-627fd75f-b404-4057-80bb-0e395c193f73 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:31.646813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-627fd75f-b404-4057-80bb-0e395c193f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:31.721716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 67/143] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:31.766547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deee31f-9594-42fd-9e05-b86ef6741ef4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:31.891493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8deee31f-9594-42fd-9e05-b86ef6741ef4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:31.904177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:31.941044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138908d0-7cea-4e5f-be2e-74c4ef3ca69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:31.942259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138908d0-7cea-4e5f-be2e-74c4ef3ca69d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:31.942740 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138908d0-7cea-4e5f-be2e-74c4ef3ca69d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:31.943188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138908d0-7cea-4e5f-be2e-74c4ef3ca69d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:31.961231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138908d0-7cea-4e5f-be2e-74c4ef3ca69d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-138908d0-7cea-4e5f-be2e-74c4ef3ca69d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:31.970253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:31.971350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:31.972492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:31.977801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 77/144] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:31.986911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fd75f-b404-4057-80bb-0e395c193f73 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:32.099228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:32.111092 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-627fd75f-b404-4057-80bb-0e395c193f73 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:32.159172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77512984-fb00-4f92-ac81-77a219d2b7d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:32.159874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77512984-fb00-4f92-ac81-77a219d2b7d2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:32.160194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77512984-fb00-4f92-ac81-77a219d2b7d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:32.160454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77512984-fb00-4f92-ac81-77a219d2b7d2 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:32.161220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77512984-fb00-4f92-ac81-77a219d2b7d2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:32.161554 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77512984-fb00-4f92-ac81-77a219d2b7d2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:32.162962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77512984-fb00-4f92-ac81-77a219d2b7d2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:32.178074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 68/145] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:32.183747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138908d0-7cea-4e5f-be2e-74c4ef3ca69d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:32.330967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138908d0-7cea-4e5f-be2e-74c4ef3ca69d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:32.380402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf89fc-75be-4121-bb9a-8da383d401ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:32.381677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf89fc-75be-4121-bb9a-8da383d401ea None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:32.382164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf89fc-75be-4121-bb9a-8da383d401ea None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:32.382635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bf89fc-75be-4121-bb9a-8da383d401ea None admin] PATH_INFO: `/v3/users/c9cf50bf3f924b6dac930c74893f5fb5` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:32.386227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf89fc-75be-4121-bb9a-8da383d401ea None admin] RBAC: Authorizing `identity:get_user(user_id=c9cf50bf3f924b6dac930c74893f5fb5)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:32.388012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bf89fc-75be-4121-bb9a-8da383d401ea None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:32.390985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 78/146] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:32 2022] GET /identity/v3/users/c9cf50bf3f924b6dac930c74893f5fb5 => generated 247 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:32.396347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77512984-fb00-4f92-ac81-77a219d2b7d2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:32.530627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77512984-fb00-4f92-ac81-77a219d2b7d2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:32.570790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd72a51-71b2-4023-8eaf-fdd650b968a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:32.571475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd72a51-71b2-4023-8eaf-fdd650b968a1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:32.571764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd72a51-71b2-4023-8eaf-fdd650b968a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:32.572043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd72a51-71b2-4023-8eaf-fdd650b968a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:32.604548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:32.663671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd72a51-71b2-4023-8eaf-fdd650b968a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:32.665166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd72a51-71b2-4023-8eaf-fdd650b968a1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:32.784896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 69/147] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:32.790511 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf89fc-75be-4121-bb9a-8da383d401ea None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:32.857364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:32.943546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bf89fc-75be-4121-bb9a-8da383d401ea None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:32.988934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a969c-575f-4e6f-93d3-b8280b76e998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:32.989884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a969c-575f-4e6f-93d3-b8280b76e998 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:32.990173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a969c-575f-4e6f-93d3-b8280b76e998 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:32.990453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a969c-575f-4e6f-93d3-b8280b76e998 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:32.994232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a969c-575f-4e6f-93d3-b8280b76e998 None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:32.995692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a969c-575f-4e6f-93d3-b8280b76e998 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:32.997546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 79/148] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:32 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:33.004189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd72a51-71b2-4023-8eaf-fdd650b968a1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:33.082896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:33.139788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd72a51-71b2-4023-8eaf-fdd650b968a1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:33.184202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c6ee21-8ea6-4d3b-93fd-51c60054ed2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:33.184884 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ee21-8ea6-4d3b-93fd-51c60054ed2f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:33.185172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ee21-8ea6-4d3b-93fd-51c60054ed2f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:33.185453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c6ee21-8ea6-4d3b-93fd-51c60054ed2f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:33.186963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6ee21-8ea6-4d3b-93fd-51c60054ed2f None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4, user.id=c9cf50bf3f924b6dac930c74893f5fb5) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:33.187284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6ee21-8ea6-4d3b-93fd-51c60054ed2f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:33.188768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c6ee21-8ea6-4d3b-93fd-51c60054ed2f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:33.198156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 70/149] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:33 2022] GET /identity/v3/role_assignments?user.id=c9cf50bf3f924b6dac930c74893f5fb5&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 608 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:33.238815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3055a7-80d1-42e8-bc80-4f41b0f4777b None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:33.239110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3055a7-80d1-42e8-bc80-4f41b0f4777b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:33.239571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3055a7-80d1-42e8-bc80-4f41b0f4777b None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:33.241116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 80/150] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:33.255493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b46894-bcbc-4c7a-8cf1-9e70088a5cbb None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:33.255787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b46894-bcbc-4c7a-8cf1-9e70088a5cbb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:33.256068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b46894-bcbc-4c7a-8cf1-9e70088a5cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:33.289588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-22b46894-bcbc-4c7a-8cf1-9e70088a5cbb None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:33.290805 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-22b46894-bcbc-4c7a-8cf1-9e70088a5cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:33.388219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 71/151] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:33.423956 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3055a7-80d1-42e8-bc80-4f41b0f4777b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:33.440994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5b108-a482-467f-b9f6-0a54f42d5930 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:33.441326 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5b108-a482-467f-b9f6-0a54f42d5930 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:33.441602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab5b108-a482-467f-b9f6-0a54f42d5930 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:33.443012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 72/152] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:33.454085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809c70-920d-41c5-85b7-b76f24e8ee79 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:33.454397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809c70-920d-41c5-85b7-b76f24e8ee79 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:33.454636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e809c70-920d-41c5-85b7-b76f24e8ee79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:33.480505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4e809c70-920d-41c5-85b7-b76f24e8ee79 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:33.486073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4e809c70-920d-41c5-85b7-b76f24e8ee79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:33.563761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:33.618252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3055a7-80d1-42e8-bc80-4f41b0f4777b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:33.620706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:33.640681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 73/153] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:33.674821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8e8d2-4964-4a38-8b80-491c9f8e4601 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:33.675647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8e8d2-4964-4a38-8b80-491c9f8e4601 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:33.675935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8e8d2-4964-4a38-8b80-491c9f8e4601 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:33.676206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f8e8d2-4964-4a38-8b80-491c9f8e4601 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:33.678584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e809c70-920d-41c5-85b7-b76f24e8ee79 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:33.688308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f8e8d2-4964-4a38-8b80-491c9f8e4601 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-33f8e8d2-4964-4a38-8b80-491c9f8e4601 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:33.696111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:33.696979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:33.697790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:33.703672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 81/154] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:33.710390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8e8d2-4964-4a38-8b80-491c9f8e4601 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:33.840485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e809c70-920d-41c5-85b7-b76f24e8ee79 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:33.895297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b933d-2299-4d9f-b976-f143f988d3fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:33.896042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b933d-2299-4d9f-b976-f143f988d3fd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:33.896326 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b933d-2299-4d9f-b976-f143f988d3fd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:33.896599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15b933d-2299-4d9f-b976-f143f988d3fd None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:33.898461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f8e8d2-4964-4a38-8b80-491c9f8e4601 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:33.914500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15b933d-2299-4d9f-b976-f143f988d3fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-c15b933d-2299-4d9f-b976-f143f988d3fd None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:33.922384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:33.923249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:33.924219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 74/155] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:33.929297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b933d-2299-4d9f-b976-f143f988d3fd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:33.988020 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af988d-b66e-4770-8104-4001248a8347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:33.989086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af988d-b66e-4770-8104-4001248a8347 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:33.989590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af988d-b66e-4770-8104-4001248a8347 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:33.990083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9af988d-b66e-4770-8104-4001248a8347 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:33.991096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af988d-b66e-4770-8104-4001248a8347 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:33.991601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af988d-b66e-4770-8104-4001248a8347 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:33.993254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9af988d-b66e-4770-8104-4001248a8347 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:33.999050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:34.019001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 82/156] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:33 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:34.023314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af988d-b66e-4770-8104-4001248a8347 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:34.089876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:34.096638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15b933d-2299-4d9f-b976-f143f988d3fd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:34.145048 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc3001-1023-41b0-ac28-db82aae3eeb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:34.145818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc3001-1023-41b0-ac28-db82aae3eeb5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:34.146040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc3001-1023-41b0-ac28-db82aae3eeb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:34.146318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bdc3001-1023-41b0-ac28-db82aae3eeb5 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:34.147070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc3001-1023-41b0-ac28-db82aae3eeb5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:34.147377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc3001-1023-41b0-ac28-db82aae3eeb5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:34.148840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bdc3001-1023-41b0-ac28-db82aae3eeb5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:34.164347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 75/157] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:34.172384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc3001-1023-41b0-ac28-db82aae3eeb5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:34.237550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9af988d-b66e-4770-8104-4001248a8347 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:34.326884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a979395-78e4-4a01-86f9-074bfa10cf33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:34.327956 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a979395-78e4-4a01-86f9-074bfa10cf33 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:34.328331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a979395-78e4-4a01-86f9-074bfa10cf33 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:34.328618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a979395-78e4-4a01-86f9-074bfa10cf33 None admin] PATH_INFO: `/v3/users/acff0c0a0c4346f697ad5aa1f0334d80` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:34.330001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a979395-78e4-4a01-86f9-074bfa10cf33 None admin] RBAC: Authorizing `identity:get_user(user_id=acff0c0a0c4346f697ad5aa1f0334d80)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:34.331495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a979395-78e4-4a01-86f9-074bfa10cf33 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:34.333534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 83/158] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:34 2022] GET /identity/v3/users/acff0c0a0c4346f697ad5aa1f0334d80 => generated 289 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:34.338935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a979395-78e4-4a01-86f9-074bfa10cf33 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:34.389872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:34.396475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bdc3001-1023-41b0-ac28-db82aae3eeb5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:34.449065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00907c-f96f-4df6-8b8f-7873d269c425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:34.449985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00907c-f96f-4df6-8b8f-7873d269c425 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:34.450385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00907c-f96f-4df6-8b8f-7873d269c425 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:34.450668 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b00907c-f96f-4df6-8b8f-7873d269c425 None admin] PATH_INFO: `/v3/users/343f2a00f3ed43acb00bd614cb69766e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:34.451962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00907c-f96f-4df6-8b8f-7873d269c425 None admin] RBAC: Authorizing `identity:get_user(user_id=343f2a00f3ed43acb00bd614cb69766e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:34.453630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b00907c-f96f-4df6-8b8f-7873d269c425 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:34.455327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 76/159] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:34 2022] GET /identity/v3/users/343f2a00f3ed43acb00bd614cb69766e => generated 275 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:34.460675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00907c-f96f-4df6-8b8f-7873d269c425 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:34.554328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:34.579513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a979395-78e4-4a01-86f9-074bfa10cf33 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:34.634930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b00907c-f96f-4df6-8b8f-7873d269c425 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:34.648067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b205fa2-b2bd-4364-b3a8-a8c0f2635eaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:34.648963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b205fa2-b2bd-4364-b3a8-a8c0f2635eaa None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:34.649249 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b205fa2-b2bd-4364-b3a8-a8c0f2635eaa None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:34.649542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b205fa2-b2bd-4364-b3a8-a8c0f2635eaa None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:34.651789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b205fa2-b2bd-4364-b3a8-a8c0f2635eaa None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:34.655283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b205fa2-b2bd-4364-b3a8-a8c0f2635eaa None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:34.657140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 84/160] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:34 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 319 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:34.662914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b205fa2-b2bd-4364-b3a8-a8c0f2635eaa None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:34.705949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca28595-a6b8-4225-aa25-834cc7a076ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:34.706620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca28595-a6b8-4225-aa25-834cc7a076ab None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:34.706910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca28595-a6b8-4225-aa25-834cc7a076ab None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:34.707186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca28595-a6b8-4225-aa25-834cc7a076ab None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:34.757619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:34.801105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca28595-a6b8-4225-aa25-834cc7a076ab None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:34.803098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca28595-a6b8-4225-aa25-834cc7a076ab None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:34.915638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b205fa2-b2bd-4364-b3a8-a8c0f2635eaa None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:34.924358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 77/161] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:09:34.926469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165551f1-1d50-4027-8645-c8425c0291cd None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:34.926909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165551f1-1d50-4027-8645-c8425c0291cd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:34.927335 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-165551f1-1d50-4027-8645-c8425c0291cd None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:34.928956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 78/162] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:34.930275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0415c-9012-47d2-9c25-a114d43923f5 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:34.930831 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0415c-9012-47d2-9c25-a114d43923f5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:34.931284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a0415c-9012-47d2-9c25-a114d43923f5 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:34.932709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 79/163] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:34.933658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0415c-9012-47d2-9c25-a114d43923f5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:34.984310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5945cdf7-c20f-404e-8399-ea1043ac460c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:34.985295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945cdf7-c20f-404e-8399-ea1043ac460c None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:34.988565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945cdf7-c20f-404e-8399-ea1043ac460c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:34.988565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5945cdf7-c20f-404e-8399-ea1043ac460c None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/users/acff0c0a0c4346f697ad5aa1f0334d80/roles/997417d78f914e27958b852313dcae0e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:34.990091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945cdf7-c20f-404e-8399-ea1043ac460c None admin] RBAC: Authorizing `identity:create_grant(project_id=a60cff69e3074f9bb2d0f231a1fb2be4, user_id=acff0c0a0c4346f697ad5aa1f0334d80, role_id=997417d78f914e27958b852313dcae0e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:34.991711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5945cdf7-c20f-404e-8399-ea1043ac460c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:35.016181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 85/164] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:34 2022] PUT /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/users/acff0c0a0c4346f697ad5aa1f0334d80/roles/997417d78f914e27958b852313dcae0e => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:09:35.019463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca2957-6e00-4579-8c55-eb4d0786a52c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.020114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca2957-6e00-4579-8c55-eb4d0786a52c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.022619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97ca2957-6e00-4579-8c55-eb4d0786a52c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.039493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:35.060661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-97ca2957-6e00-4579-8c55-eb4d0786a52c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:35.063899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-97ca2957-6e00-4579-8c55-eb4d0786a52c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:35.103995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:35.116382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a0415c-9012-47d2-9c25-a114d43923f5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:35.188517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 86/165] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:35.190144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46c88e-02f5-4ea3-872c-217b52b28662 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.190445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46c88e-02f5-4ea3-872c-217b52b28662 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.190763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa46c88e-02f5-4ea3-872c-217b52b28662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.194776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eac4c9-c175-4116-9f25-2747ac3ebe90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:35.195692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eac4c9-c175-4116-9f25-2747ac3ebe90 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.195968 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eac4c9-c175-4116-9f25-2747ac3ebe90 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.196240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eac4c9-c175-4116-9f25-2747ac3ebe90 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.197496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eac4c9-c175-4116-9f25-2747ac3ebe90 None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:35.198976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eac4c9-c175-4116-9f25-2747ac3ebe90 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:35.200910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 80/166] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:34 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:35.210271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-aa46c88e-02f5-4ea3-872c-217b52b28662 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:35.210470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eac4c9-c175-4116-9f25-2747ac3ebe90 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:35.213971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-aa46c88e-02f5-4ea3-872c-217b52b28662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:35.287437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 87/167] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:35.288308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46c88e-02f5-4ea3-872c-217b52b28662 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:35.406623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:35.415494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eac4c9-c175-4116-9f25-2747ac3ebe90 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:35.426851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa46c88e-02f5-4ea3-872c-217b52b28662 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:35.464978 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffb443-0668-4dfd-9c03-3fda9bde37f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:35.465736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffb443-0668-4dfd-9c03-3fda9bde37f9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.466022 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffb443-0668-4dfd-9c03-3fda9bde37f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.466308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ffb443-0668-4dfd-9c03-3fda9bde37f9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.477567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ffb443-0668-4dfd-9c03-3fda9bde37f9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f0ffb443-0668-4dfd-9c03-3fda9bde37f9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:35.485847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:35.487031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:35.487849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 88/168] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:09:35 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:35.488497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffb443-0668-4dfd-9c03-3fda9bde37f9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:35.558830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b16d47-c067-4f6f-8fc2-b3117e28a2a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:35.559518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b16d47-c067-4f6f-8fc2-b3117e28a2a0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.559800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b16d47-c067-4f6f-8fc2-b3117e28a2a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.560067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b16d47-c067-4f6f-8fc2-b3117e28a2a0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.561564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b16d47-c067-4f6f-8fc2-b3117e28a2a0 None admin] RBAC: Adding query filter params (role.id=91f9d5c6716547c18da989bf327f6aca, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4, user.id=343f2a00f3ed43acb00bd614cb69766e) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:35.561835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b16d47-c067-4f6f-8fc2-b3117e28a2a0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:35.563294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75b16d47-c067-4f6f-8fc2-b3117e28a2a0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:35.565818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:35.583536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 81/169] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:35 2022] GET /identity/v3/role_assignments?user.id=343f2a00f3ed43acb00bd614cb69766e&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=91f9d5c6716547c18da989bf327f6aca => generated 608 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:35.584386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b16d47-c067-4f6f-8fc2-b3117e28a2a0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:35.617694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ffb443-0668-4dfd-9c03-3fda9bde37f9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:35.680717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9b7177-84a5-460b-960d-5188e916b0e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:35.681559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b7177-84a5-460b-960d-5188e916b0e6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.681852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b7177-84a5-460b-960d-5188e916b0e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.682131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9b7177-84a5-460b-960d-5188e916b0e6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.694666 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9b7177-84a5-460b-960d-5188e916b0e6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-be9b7177-84a5-460b-960d-5188e916b0e6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:35.706791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:35.707834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:35.708644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 89/170] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:35 2022] GET /identity/v3/roles/member => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:35.710150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0207-815d-4255-b388-fd6c3ad6c095 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.710150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0207-815d-4255-b388-fd6c3ad6c095 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.710150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce4f0207-815d-4255-b388-fd6c3ad6c095 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.711359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 90/171] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:35.712451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d283dd8-19ac-4d32-89da-8f81f325464e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.712731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d283dd8-19ac-4d32-89da-8f81f325464e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.713001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d283dd8-19ac-4d32-89da-8f81f325464e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.714270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 91/172] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:35 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:35.719158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5723dc-da2d-46c6-8205-de385ba472ee None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.719368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5723dc-da2d-46c6-8205-de385ba472ee None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.719650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5723dc-da2d-46c6-8205-de385ba472ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.726295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:35.740899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5b5723dc-da2d-46c6-8205-de385ba472ee None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:35.742247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5b5723dc-da2d-46c6-8205-de385ba472ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:35.817254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 92/173] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:35.818032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5723dc-da2d-46c6-8205-de385ba472ee None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:35.875622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b16d47-c067-4f6f-8fc2-b3117e28a2a0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:35.919322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ff4b9b-44a6-44eb-841c-f8f113bda9d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:35.921259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff4b9b-44a6-44eb-841c-f8f113bda9d4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.921810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff4b9b-44a6-44eb-841c-f8f113bda9d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.922269 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ff4b9b-44a6-44eb-841c-f8f113bda9d4 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.923209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff4b9b-44a6-44eb-841c-f8f113bda9d4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:35.924760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff4b9b-44a6-44eb-841c-f8f113bda9d4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:35.926954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ff4b9b-44a6-44eb-841c-f8f113bda9d4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:35.936984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 82/174] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:09:35 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 353 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:35.938739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6c7e9-2f2f-41ac-bca7-d313648bed91 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:35.939255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6c7e9-2f2f-41ac-bca7-d313648bed91 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:35.939737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e6c7e9-2f2f-41ac-bca7-d313648bed91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:35.940224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5723dc-da2d-46c6-8205-de385ba472ee None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:35.960540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-53e6c7e9-2f2f-41ac-bca7-d313648bed91 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:35.962267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-53e6c7e9-2f2f-41ac-bca7-d313648bed91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:35.993599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:36.001270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e4e42-6d13-4302-81d2-b107d7829934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.002011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e4e42-6d13-4302-81d2-b107d7829934 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.002267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e4e42-6d13-4302-81d2-b107d7829934 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.002545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1e4e42-6d13-4302-81d2-b107d7829934 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.003303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e4e42-6d13-4302-81d2-b107d7829934 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:36.003839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e4e42-6d13-4302-81d2-b107d7829934 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:36.005007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1e4e42-6d13-4302-81d2-b107d7829934 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:36.014023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 93/175] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:35 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:36.015452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e4e42-6d13-4302-81d2-b107d7829934 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:36.038178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:36.051588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 83/176] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:36.052873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6c7e9-2f2f-41ac-bca7-d313648bed91 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:36.150244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1e4e42-6d13-4302-81d2-b107d7829934 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:36.170948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e6c7e9-2f2f-41ac-bca7-d313648bed91 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:36.189894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e5265-0514-496e-b662-e381fff2550e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.191330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e5265-0514-496e-b662-e381fff2550e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.191622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e5265-0514-496e-b662-e381fff2550e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.191901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992e5265-0514-496e-b662-e381fff2550e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.203494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992e5265-0514-496e-b662-e381fff2550e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:36.207635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dfd85d-e165-4cb6-9274-7ca7cf0bd242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.208516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfd85d-e165-4cb6-9274-7ca7cf0bd242 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.208802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfd85d-e165-4cb6-9274-7ca7cf0bd242 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.209077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dfd85d-e165-4cb6-9274-7ca7cf0bd242 None admin] PATH_INFO: `/v3/users/343f2a00f3ed43acb00bd614cb69766e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.210303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfd85d-e165-4cb6-9274-7ca7cf0bd242 None admin] RBAC: Authorizing `identity:get_user(user_id=343f2a00f3ed43acb00bd614cb69766e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-992e5265-0514-496e-b662-e381fff2550e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:36.211038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:36.211863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:36.212643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:36.213227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92dfd85d-e165-4cb6-9274-7ca7cf0bd242 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:36.213292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 94/177] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:36 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:36.213292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e5265-0514-496e-b662-e381fff2550e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:36.213426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 84/178] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:36 2022] GET /identity/v3/users/343f2a00f3ed43acb00bd614cb69766e => generated 275 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:36.219373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8be603-7152-496e-be9e-23dbe37ba4fa None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.219373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8be603-7152-496e-be9e-23dbe37ba4fa None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.219373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8be603-7152-496e-be9e-23dbe37ba4fa None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.219373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 85/179] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:36 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:36.219373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8be603-7152-496e-be9e-23dbe37ba4fa None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:36.277316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:36.285207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:36.330021 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992e5265-0514-496e-b662-e381fff2550e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:36.341177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8be603-7152-496e-be9e-23dbe37ba4fa None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:36.366114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63abe3-785d-45af-96ab-db745b30563c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.367253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63abe3-785d-45af-96ab-db745b30563c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.367539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63abe3-785d-45af-96ab-db745b30563c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.367822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e63abe3-785d-45af-96ab-db745b30563c None admin] PATH_INFO: `/v3/users/343f2a00f3ed43acb00bd614cb69766e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.369085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63abe3-785d-45af-96ab-db745b30563c None admin] RBAC: Authorizing `identity:get_user(user_id=343f2a00f3ed43acb00bd614cb69766e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:36.370681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e63abe3-785d-45af-96ab-db745b30563c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:36.372560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 95/180] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:36 2022] GET /identity/v3/users/343f2a00f3ed43acb00bd614cb69766e => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:36.373340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63abe3-785d-45af-96ab-db745b30563c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:36.387440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0d4960-02b7-4fe6-970d-52f38eef8560 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.388593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d4960-02b7-4fe6-970d-52f38eef8560 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.388923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d4960-02b7-4fe6-970d-52f38eef8560 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.389196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0d4960-02b7-4fe6-970d-52f38eef8560 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.390957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d4960-02b7-4fe6-970d-52f38eef8560 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:36.392862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0d4960-02b7-4fe6-970d-52f38eef8560 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:36.399295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 86/181] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:36.399295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a0e3-eaea-4ad4-b57d-df46a40b5284 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.399295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a0e3-eaea-4ad4-b57d-df46a40b5284 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.399295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3098a0e3-eaea-4ad4-b57d-df46a40b5284 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.419073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3098a0e3-eaea-4ad4-b57d-df46a40b5284 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:36.421103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3098a0e3-eaea-4ad4-b57d-df46a40b5284 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:36.480489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e63abe3-785d-45af-96ab-db745b30563c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:36.501413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 87/182] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:36.502327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098a0e3-eaea-4ad4-b57d-df46a40b5284 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:36.512895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:36.525021 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc37e3e-fbe3-4752-9fea-6ddcdc97c506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.525502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc37e3e-fbe3-4752-9fea-6ddcdc97c506 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.525827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc37e3e-fbe3-4752-9fea-6ddcdc97c506 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.526103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc37e3e-fbe3-4752-9fea-6ddcdc97c506 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.565328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:36.606207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc37e3e-fbe3-4752-9fea-6ddcdc97c506 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:36.608229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc37e3e-fbe3-4752-9fea-6ddcdc97c506 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:36.612904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3098a0e3-eaea-4ad4-b57d-df46a40b5284 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:36.647310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35858c-fa61-4ab7-ab46-e32e37a07630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.647981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35858c-fa61-4ab7-ab46-e32e37a07630 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.648264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35858c-fa61-4ab7-ab46-e32e37a07630 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.648542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb35858c-fa61-4ab7-ab46-e32e37a07630 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.649282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35858c-fa61-4ab7-ab46-e32e37a07630 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:36.649621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35858c-fa61-4ab7-ab46-e32e37a07630 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:36.651180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb35858c-fa61-4ab7-ab46-e32e37a07630 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:36.664302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 88/183] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:36 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:36.664302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35858c-fa61-4ab7-ab46-e32e37a07630 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:36.717229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 96/184] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:36.718407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc37e3e-fbe3-4752-9fea-6ddcdc97c506 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:36.746827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:36.769569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb35858c-fa61-4ab7-ab46-e32e37a07630 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:36.787200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:36.805846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa2a1-bda6-43b5-ac91-b37830931dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.806505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa2a1-bda6-43b5-ac91-b37830931dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.806791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa2a1-bda6-43b5-ac91-b37830931dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.807069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5aa2a1-bda6-43b5-ac91-b37830931dc2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.833317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc37e3e-fbe3-4752-9fea-6ddcdc97c506 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:36.871328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce0595-2381-4ef3-a169-83b9abcf8e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:36.872138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0595-2381-4ef3-a169-83b9abcf8e2f None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:36.872423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0595-2381-4ef3-a169-83b9abcf8e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:36.872923 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bce0595-2381-4ef3-a169-83b9abcf8e2f None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:36.874257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce0595-2381-4ef3-a169-83b9abcf8e2f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:36.875723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bce0595-2381-4ef3-a169-83b9abcf8e2f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:36.879936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa2a1-bda6-43b5-ac91-b37830931dc2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:36.881397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e5aa2a1-bda6-43b5-ac91-b37830931dc2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:36.913540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 97/185] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:09:36 2022] POST /identity/v3/users => generated 297 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:36.913540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce0595-2381-4ef3-a169-83b9abcf8e2f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:37.017320 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 89/186] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:37.018288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa2a1-bda6-43b5-ac91-b37830931dc2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:37.043138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:37.048985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:37.072846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bce0595-2381-4ef3-a169-83b9abcf8e2f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:37.115149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9e893-7f97-4588-865b-b2a1df29c026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:37.116191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9e893-7f97-4588-865b-b2a1df29c026 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:37.116621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9e893-7f97-4588-865b-b2a1df29c026 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:37.117041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca9e893-7f97-4588-865b-b2a1df29c026 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:37.130937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca9e893-7f97-4588-865b-b2a1df29c026 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-8ca9e893-7f97-4588-865b-b2a1df29c026 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:37.139449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:37.140394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:37.141189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 98/187] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:09:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:37.141875 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9e893-7f97-4588-865b-b2a1df29c026 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:37.152803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5aa2a1-bda6-43b5-ac91-b37830931dc2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:37.196921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a1897-26c1-407f-a217-ad5cf2a75ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:37.197980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a1897-26c1-407f-a217-ad5cf2a75ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:37.198261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a1897-26c1-407f-a217-ad5cf2a75ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:37.198533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761a1897-26c1-407f-a217-ad5cf2a75ac6 None admin] PATH_INFO: `/v3/users/0743416a65dd4d699a6b6695f3076fc0` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:37.218947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a1897-26c1-407f-a217-ad5cf2a75ac6 None admin] RBAC: Authorizing `identity:get_user(user_id=0743416a65dd4d699a6b6695f3076fc0)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:37.220454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-761a1897-26c1-407f-a217-ad5cf2a75ac6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:37.222240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 90/188] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:37 2022] GET /identity/v3/users/0743416a65dd4d699a6b6695f3076fc0 => generated 283 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:37.222999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a1897-26c1-407f-a217-ad5cf2a75ac6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:37.287740 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca9e893-7f97-4588-865b-b2a1df29c026 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:37.320891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:37.336213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c7036-05a0-43ff-b129-9a9bf1582316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:37.337120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c7036-05a0-43ff-b129-9a9bf1582316 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:37.337408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c7036-05a0-43ff-b129-9a9bf1582316 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:37.337708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26c7036-05a0-43ff-b129-9a9bf1582316 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:37.339977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c7036-05a0-43ff-b129-9a9bf1582316 None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:37.341480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26c7036-05a0-43ff-b129-9a9bf1582316 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:37.343439 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 99/189] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:37 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:37.344137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c7036-05a0-43ff-b129-9a9bf1582316 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:37.352699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761a1897-26c1-407f-a217-ad5cf2a75ac6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:37.354745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:37.408725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14537c8-1292-4878-820a-46137cd28cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:37.409626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14537c8-1292-4878-820a-46137cd28cfb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:37.409909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14537c8-1292-4878-820a-46137cd28cfb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:37.410181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14537c8-1292-4878-820a-46137cd28cfb None admin] PATH_INFO: `/v3/projects/13ffb9be5ed64a9bb86d078f371e308b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:37.415575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14537c8-1292-4878-820a-46137cd28cfb None admin] RBAC: Authorizing `identity:get_project(project_id=13ffb9be5ed64a9bb86d078f371e308b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:37.417038 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14537c8-1292-4878-820a-46137cd28cfb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:37.418794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 91/190] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:37 2022] GET /identity/v3/projects/13ffb9be5ed64a9bb86d078f371e308b => generated 312 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:37.422266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14537c8-1292-4878-820a-46137cd28cfb None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:37.535533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26c7036-05a0-43ff-b129-9a9bf1582316 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:37.582502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14537c8-1292-4878-820a-46137cd28cfb None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:37.583639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968c4bf-e6ff-4bf2-9cee-15a174b01de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:37.584580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968c4bf-e6ff-4bf2-9cee-15a174b01de2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:37.585811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968c4bf-e6ff-4bf2-9cee-15a174b01de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:37.586097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8968c4bf-e6ff-4bf2-9cee-15a174b01de2 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:37.586858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968c4bf-e6ff-4bf2-9cee-15a174b01de2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:37.587171 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968c4bf-e6ff-4bf2-9cee-15a174b01de2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:37.588584 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8968c4bf-e6ff-4bf2-9cee-15a174b01de2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:37.601332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 100/191] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:09:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:37.602261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968c4bf-e6ff-4bf2-9cee-15a174b01de2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:37.624452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fec9c7-235d-48f3-832b-4ad563ed30a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:37.625107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fec9c7-235d-48f3-832b-4ad563ed30a5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:37.625571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fec9c7-235d-48f3-832b-4ad563ed30a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:37.625854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fec9c7-235d-48f3-832b-4ad563ed30a5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:37.643422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:37.706041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fec9c7-235d-48f3-832b-4ad563ed30a5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:37.707752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17fec9c7-235d-48f3-832b-4ad563ed30a5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:37.709486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:37.717443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8968c4bf-e6ff-4bf2-9cee-15a174b01de2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:37.753973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59eb592-9697-479d-a458-470f8405e01b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:37.754642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eb592-9697-479d-a458-470f8405e01b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:37.754930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eb592-9697-479d-a458-470f8405e01b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:37.755211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59eb592-9697-479d-a458-470f8405e01b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:37.756155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59eb592-9697-479d-a458-470f8405e01b None admin] RBAC: Adding query filter params (role.id=77052d58af4f4b5d9b2d003531462e50, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4, user.id=343f2a00f3ed43acb00bd614cb69766e) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:37.756476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59eb592-9697-479d-a458-470f8405e01b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:37.758198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59eb592-9697-479d-a458-470f8405e01b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:37.765595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 101/192] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:37 2022] GET /identity/v3/role_assignments?user.id=343f2a00f3ed43acb00bd614cb69766e&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=77052d58af4f4b5d9b2d003531462e50 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:37.766378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59eb592-9697-479d-a458-470f8405e01b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:37.816636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 92/193] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:37.822466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3305009-c574-41d2-8639-274ebcbbb876 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:37.822763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3305009-c574-41d2-8639-274ebcbbb876 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:37.823044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3305009-c574-41d2-8639-274ebcbbb876 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:37.824687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 93/194] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:37.825280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3305009-c574-41d2-8639-274ebcbbb876 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:37.893123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59eb592-9697-479d-a458-470f8405e01b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:37.929840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:37.940152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4379b13-cdc2-49a0-92d3-341af63d19fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:37.940831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4379b13-cdc2-49a0-92d3-341af63d19fe None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:37.941116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4379b13-cdc2-49a0-92d3-341af63d19fe None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:37.941393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4379b13-cdc2-49a0-92d3-341af63d19fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:37.945358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4379b13-cdc2-49a0-92d3-341af63d19fe None admin] RBAC: Adding query filter params (role.id=91f9d5c6716547c18da989bf327f6aca, scope.project.id=13ffb9be5ed64a9bb86d078f371e308b, user.id=343f2a00f3ed43acb00bd614cb69766e) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:37.945705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4379b13-cdc2-49a0-92d3-341af63d19fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:37.947216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4379b13-cdc2-49a0-92d3-341af63d19fe None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:37.966307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 102/195] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:37 2022] GET /identity/v3/role_assignments?user.id=343f2a00f3ed43acb00bd614cb69766e&scope.project.id=13ffb9be5ed64a9bb86d078f371e308b&role.id=91f9d5c6716547c18da989bf327f6aca => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:37.967727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4379b13-cdc2-49a0-92d3-341af63d19fe None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:37.988795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:38.061690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3305009-c574-41d2-8639-274ebcbbb876 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:38.103758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe5a078-82f5-47b3-bb05-3d33b649bf17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:38.104652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5a078-82f5-47b3-bb05-3d33b649bf17 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.104939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5a078-82f5-47b3-bb05-3d33b649bf17 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.105221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe5a078-82f5-47b3-bb05-3d33b649bf17 None admin] PATH_INFO: `/v3/users/0743416a65dd4d699a6b6695f3076fc0` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:38.106536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5a078-82f5-47b3-bb05-3d33b649bf17 None admin] RBAC: Authorizing `identity:get_user(user_id=0743416a65dd4d699a6b6695f3076fc0)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:38.108305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe5a078-82f5-47b3-bb05-3d33b649bf17 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:38.111063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 94/196] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:37 2022] GET /identity/v3/users/0743416a65dd4d699a6b6695f3076fc0 => generated 283 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:38.113075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08523193-8428-46a7-ba8f-e9d2a6815a13 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.113582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08523193-8428-46a7-ba8f-e9d2a6815a13 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.113905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08523193-8428-46a7-ba8f-e9d2a6815a13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:38.115444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4379b13-cdc2-49a0-92d3-341af63d19fe None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:38.133351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-08523193-8428-46a7-ba8f-e9d2a6815a13 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:38.134639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-08523193-8428-46a7-ba8f-e9d2a6815a13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:38.163600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f55e2d-6309-4a08-99fc-ad3cacbb5d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:38.164485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f55e2d-6309-4a08-99fc-ad3cacbb5d9e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.164772 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f55e2d-6309-4a08-99fc-ad3cacbb5d9e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.165051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f55e2d-6309-4a08-99fc-ad3cacbb5d9e None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:38.169369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f55e2d-6309-4a08-99fc-ad3cacbb5d9e None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:38.171231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f55e2d-6309-4a08-99fc-ad3cacbb5d9e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:38.174467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 103/197] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:37 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:38.176848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f55e2d-6309-4a08-99fc-ad3cacbb5d9e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:38.264401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 95/198] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:38.265014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08523193-8428-46a7-ba8f-e9d2a6815a13 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:38.309528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:38.354814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:38.370238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f55e2d-6309-4a08-99fc-ad3cacbb5d9e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:38.409069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a19b-512a-4824-af57-bea959bee393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:38.409757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a19b-512a-4824-af57-bea959bee393 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.410045 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a19b-512a-4824-af57-bea959bee393 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.410528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b59a19b-512a-4824-af57-bea959bee393 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:38.422887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08523193-8428-46a7-ba8f-e9d2a6815a13 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:38.468753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea4e54-befc-4381-bc66-9958438c0263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:38.469633 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea4e54-befc-4381-bc66-9958438c0263 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.469922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea4e54-befc-4381-bc66-9958438c0263 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.470202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ea4e54-befc-4381-bc66-9958438c0263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:38.471744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea4e54-befc-4381-bc66-9958438c0263 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=edc462e56b784f1eaefd9e0b56312647, user.id=0743416a65dd4d699a6b6695f3076fc0) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:38.472069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea4e54-befc-4381-bc66-9958438c0263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:38.474246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ea4e54-befc-4381-bc66-9958438c0263 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:38.483907 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 96/199] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:38 2022] GET /identity/v3/role_assignments?user.id=0743416a65dd4d699a6b6695f3076fc0&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 259 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:38.484901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea4e54-befc-4381-bc66-9958438c0263 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:38.495110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59a19b-512a-4824-af57-bea959bee393 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:38.496683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b59a19b-512a-4824-af57-bea959bee393 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:38.615145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ea4e54-befc-4381-bc66-9958438c0263 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:38.625096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:38.639814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:38.660600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f217ece-4db7-48bc-9f7d-643df9bf95ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:38.661858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f217ece-4db7-48bc-9f7d-643df9bf95ff None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.662314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f217ece-4db7-48bc-9f7d-643df9bf95ff None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.662738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f217ece-4db7-48bc-9f7d-643df9bf95ff None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:38.677406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f217ece-4db7-48bc-9f7d-643df9bf95ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6f217ece-4db7-48bc-9f7d-643df9bf95ff None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:38.687135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:38.688198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:38.688993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:38.692416 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 97/200] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:38 2022] GET /identity/v3/roles/reader => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:38.698691 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f217ece-4db7-48bc-9f7d-643df9bf95ff None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:38.736187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 104/201] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 561 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:38.738756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a19b-512a-4824-af57-bea959bee393 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:38.816404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f217ece-4db7-48bc-9f7d-643df9bf95ff None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:38.874340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531ed52-946d-44c4-8e4f-54ce590d0198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:38.874942 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ed52-946d-44c4-8e4f-54ce590d0198 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:38.875229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ed52-946d-44c4-8e4f-54ce590d0198 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:38.875506 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7531ed52-946d-44c4-8e4f-54ce590d0198 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:38.876267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531ed52-946d-44c4-8e4f-54ce590d0198 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:38.876576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531ed52-946d-44c4-8e4f-54ce590d0198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:38.877999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7531ed52-946d-44c4-8e4f-54ce590d0198 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:38.895833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 98/202] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:38.901179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531ed52-946d-44c4-8e4f-54ce590d0198 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:38.927141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b59a19b-512a-4824-af57-bea959bee393 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:38.960186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.001213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d92472-13fb-4c1c-bcbc-b2f7afe0be15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.002121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d92472-13fb-4c1c-bcbc-b2f7afe0be15 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.002405 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d92472-13fb-4c1c-bcbc-b2f7afe0be15 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.002684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d92472-13fb-4c1c-bcbc-b2f7afe0be15 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.006873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d92472-13fb-4c1c-bcbc-b2f7afe0be15 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.008364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64d92472-13fb-4c1c-bcbc-b2f7afe0be15 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.010145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 105/203] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:38 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:39.016872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d92472-13fb-4c1c-bcbc-b2f7afe0be15 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.081106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.103198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7531ed52-946d-44c4-8e4f-54ce590d0198 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.155465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e15f7-aae1-4b9a-9b4a-03dc773b9201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.156352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e15f7-aae1-4b9a-9b4a-03dc773b9201 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.156635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e15f7-aae1-4b9a-9b4a-03dc773b9201 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.156914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6e15f7-aae1-4b9a-9b4a-03dc773b9201 None admin] PATH_INFO: `/v3/users/acff0c0a0c4346f697ad5aa1f0334d80` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.158278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e15f7-aae1-4b9a-9b4a-03dc773b9201 None admin] RBAC: Authorizing `identity:get_user(user_id=acff0c0a0c4346f697ad5aa1f0334d80)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.159753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb6e15f7-aae1-4b9a-9b4a-03dc773b9201 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.165613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 99/204] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:38 2022] GET /identity/v3/users/acff0c0a0c4346f697ad5aa1f0334d80 => generated 289 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:39.167437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e15f7-aae1-4b9a-9b4a-03dc773b9201 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.183729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d92472-13fb-4c1c-bcbc-b2f7afe0be15 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.224026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09000fc1-6038-4b3f-95d4-997ebe71934b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.225022 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09000fc1-6038-4b3f-95d4-997ebe71934b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.225322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09000fc1-6038-4b3f-95d4-997ebe71934b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.228993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09000fc1-6038-4b3f-95d4-997ebe71934b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.230105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09000fc1-6038-4b3f-95d4-997ebe71934b None admin] RBAC: Adding query filter params (role.id=77052d58af4f4b5d9b2d003531462e50, scope.project.id=edc462e56b784f1eaefd9e0b56312647, user.id=0743416a65dd4d699a6b6695f3076fc0) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:39.230620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09000fc1-6038-4b3f-95d4-997ebe71934b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.232545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09000fc1-6038-4b3f-95d4-997ebe71934b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.240952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 106/205] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/role_assignments?user.id=0743416a65dd4d699a6b6695f3076fc0&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=77052d58af4f4b5d9b2d003531462e50 => generated 259 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:39.341435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.352588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6e15f7-aae1-4b9a-9b4a-03dc773b9201 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.394062 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1062cabb-2dbb-4791-9005-a70df970300f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.395103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062cabb-2dbb-4791-9005-a70df970300f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.395628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062cabb-2dbb-4791-9005-a70df970300f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.396100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1062cabb-2dbb-4791-9005-a70df970300f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.505620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062cabb-2dbb-4791-9005-a70df970300f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.507614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1062cabb-2dbb-4791-9005-a70df970300f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.582364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014de851-2613-41c2-b4c3-eeab0979f722 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.582801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014de851-2613-41c2-b4c3-eeab0979f722 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.582966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014de851-2613-41c2-b4c3-eeab0979f722 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.584318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 107/206] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:39.593278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b9a8a-dd8d-429b-aa81-59c41425b379 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.593590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b9a8a-dd8d-429b-aa81-59c41425b379 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.593870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b9a8a-dd8d-429b-aa81-59c41425b379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.618919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-566b9a8a-dd8d-429b-aa81-59c41425b379 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:39.620129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-566b9a8a-dd8d-429b-aa81-59c41425b379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:39.655432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.663521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 100/207] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 492 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:09:39.665151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f6ae4-c27f-425f-8012-4a8322468c82 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.665693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f6ae4-c27f-425f-8012-4a8322468c82 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.666129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9f6ae4-c27f-425f-8012-4a8322468c82 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.672998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 101/208] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:39.674003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f6ae4-c27f-425f-8012-4a8322468c82 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.714518 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 108/209] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:39.715958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c0fad-158c-426c-950f-52c02d53ed2d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.721878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c0fad-158c-426c-950f-52c02d53ed2d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.722327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339c0fad-158c-426c-950f-52c02d53ed2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.752738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-339c0fad-158c-426c-950f-52c02d53ed2d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:39.753969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-339c0fad-158c-426c-950f-52c02d53ed2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:39.778026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.805166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b9f6ae4-c27f-425f-8012-4a8322468c82 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.825496 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 109/210] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:39.827123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c0fad-158c-426c-950f-52c02d53ed2d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.845121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e5b81-216e-42b6-b4aa-1576c4aff3de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.846144 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e5b81-216e-42b6-b4aa-1576c4aff3de None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.846532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e5b81-216e-42b6-b4aa-1576c4aff3de None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.846867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47e5b81-216e-42b6-b4aa-1576c4aff3de None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.848139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e5b81-216e-42b6-b4aa-1576c4aff3de None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:39.850101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f47e5b81-216e-42b6-b4aa-1576c4aff3de None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:39.852219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 102/211] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:39.857718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e5b81-216e-42b6-b4aa-1576c4aff3de None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:39.927513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:39.931795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339c0fad-158c-426c-950f-52c02d53ed2d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.971754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528e321-08ee-43cf-9daa-72a9e5596e4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:39.972573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528e321-08ee-43cf-9daa-72a9e5596e4c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:39.972861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528e321-08ee-43cf-9daa-72a9e5596e4c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:39.973139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f528e321-08ee-43cf-9daa-72a9e5596e4c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:39.975519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47e5b81-216e-42b6-b4aa-1576c4aff3de None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:39.985235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f528e321-08ee-43cf-9daa-72a9e5596e4c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f528e321-08ee-43cf-9daa-72a9e5596e4c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:39.993052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:39.994071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:39.995092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:39.996235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 110/212] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:40.001074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528e321-08ee-43cf-9daa-72a9e5596e4c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:40.021487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:40.028980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ab9df-2707-45f8-9a6e-548771945a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:40.029788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ab9df-2707-45f8-9a6e-548771945a94 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:40.030172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ab9df-2707-45f8-9a6e-548771945a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:40.030452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5ab9df-2707-45f8-9a6e-548771945a94 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:40.031352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ab9df-2707-45f8-9a6e-548771945a94 None admin] RBAC: Adding query filter params (role.id=997417d78f914e27958b852313dcae0e, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4, user.id=acff0c0a0c4346f697ad5aa1f0334d80) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:40.031718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ab9df-2707-45f8-9a6e-548771945a94 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:40.033418 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f5ab9df-2707-45f8-9a6e-548771945a94 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:40.040674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 103/213] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/role_assignments?user.id=acff0c0a0c4346f697ad5aa1f0334d80&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=997417d78f914e27958b852313dcae0e => generated 608 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:40.041279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ab9df-2707-45f8-9a6e-548771945a94 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:40.110484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f528e321-08ee-43cf-9daa-72a9e5596e4c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:40.149017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640f61-f1fd-46e3-912c-1f18097b0663 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:40.150038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640f61-f1fd-46e3-912c-1f18097b0663 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:40.150334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640f61-f1fd-46e3-912c-1f18097b0663 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:40.150615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49640f61-f1fd-46e3-912c-1f18097b0663 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:40.151915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49640f61-f1fd-46e3-912c-1f18097b0663 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:40.153436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49640f61-f1fd-46e3-912c-1f18097b0663 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:40.156152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 111/214] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:39 2022] GET /identity/v3/domains/default => generated 199 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:40.161808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640f61-f1fd-46e3-912c-1f18097b0663 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:40.253324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5ab9df-2707-45f8-9a6e-548771945a94 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:40.304723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:40.309354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:40.329870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41476af-a4ad-4129-bcb6-a613d123942c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:40.331124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41476af-a4ad-4129-bcb6-a613d123942c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:40.331348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41476af-a4ad-4129-bcb6-a613d123942c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:40.331631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41476af-a4ad-4129-bcb6-a613d123942c None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:40.332539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41476af-a4ad-4129-bcb6-a613d123942c None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:40.332948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41476af-a4ad-4129-bcb6-a613d123942c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:40.334386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41476af-a4ad-4129-bcb6-a613d123942c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:40.335107 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49640f61-f1fd-46e3-912c-1f18097b0663 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:40.345454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 104/215] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 304 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:40.351686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41476af-a4ad-4129-bcb6-a613d123942c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:40.388140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d9276-7b8e-4b59-a6a9-9b3b64b81774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:40.388898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d9276-7b8e-4b59-a6a9-9b3b64b81774 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:40.389185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d9276-7b8e-4b59-a6a9-9b3b64b81774 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:40.389553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760d9276-7b8e-4b59-a6a9-9b3b64b81774 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:40.390233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d9276-7b8e-4b59-a6a9-9b3b64b81774 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:40.391695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760d9276-7b8e-4b59-a6a9-9b3b64b81774 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:40.425641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 112/216] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:09:40 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:40.493236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41476af-a4ad-4129-bcb6-a613d123942c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:40.539924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21614ee-0e72-4f34-b971-dc792487322b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:40.540996 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21614ee-0e72-4f34-b971-dc792487322b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:40.541282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21614ee-0e72-4f34-b971-dc792487322b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:40.541689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21614ee-0e72-4f34-b971-dc792487322b None admin] PATH_INFO: `/v3/users/d96eadfaf0ba4a5bada5ae72e66ec0f6` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:40.561538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21614ee-0e72-4f34-b971-dc792487322b None admin] RBAC: Authorizing `identity:get_user(user_id=d96eadfaf0ba4a5bada5ae72e66ec0f6)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:40.563229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21614ee-0e72-4f34-b971-dc792487322b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:40.565414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 105/217] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:40 2022] GET /identity/v3/users/d96eadfaf0ba4a5bada5ae72e66ec0f6 => generated 297 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:40.570840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d9276-7b8e-4b59-a6a9-9b3b64b81774 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:40.691925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760d9276-7b8e-4b59-a6a9-9b3b64b81774 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:40.728312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba61009-5632-409a-b4a9-ae7944187cf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:40.729342 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba61009-5632-409a-b4a9-ae7944187cf7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:40.729851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba61009-5632-409a-b4a9-ae7944187cf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:40.730353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba61009-5632-409a-b4a9-ae7944187cf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:40.744150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:40.816380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba61009-5632-409a-b4a9-ae7944187cf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:40.817806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba61009-5632-409a-b4a9-ae7944187cf7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:40.934182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 113/218] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:40.940260 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21614ee-0e72-4f34-b971-dc792487322b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:40.955272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c69ff8-ab78-42ae-bba5-a71400e87405 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:40.955889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c69ff8-ab78-42ae-bba5-a71400e87405 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:40.956361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c69ff8-ab78-42ae-bba5-a71400e87405 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:40.958977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 114/219] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:40.971150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066a424-c441-4785-94dc-feb0c3e8c93d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:40.971839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066a424-c441-4785-94dc-feb0c3e8c93d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:40.972291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3066a424-c441-4785-94dc-feb0c3e8c93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:40.999444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3066a424-c441-4785-94dc-feb0c3e8c93d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:41.000917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3066a424-c441-4785-94dc-feb0c3e8c93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:41.008582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:41.064970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21614ee-0e72-4f34-b971-dc792487322b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:41.095842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:41.103006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa58fc-7695-456f-b069-7a8d9082554a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:41.104182 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa58fc-7695-456f-b069-7a8d9082554a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:41.104667 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa58fc-7695-456f-b069-7a8d9082554a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:41.105161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aa58fc-7695-456f-b069-7a8d9082554a None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:41.106597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa58fc-7695-456f-b069-7a8d9082554a None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:41.108468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aa58fc-7695-456f-b069-7a8d9082554a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:41.112440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 106/220] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:40 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:41.118132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa58fc-7695-456f-b069-7a8d9082554a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:41.157079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 115/221] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:41.200341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066a424-c441-4785-94dc-feb0c3e8c93d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:41.317613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aa58fc-7695-456f-b069-7a8d9082554a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:41.319249 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3066a424-c441-4785-94dc-feb0c3e8c93d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:41.364179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:41.369640 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ab523-a507-46ee-96fe-5cf7fa0be18f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:41.370714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ab523-a507-46ee-96fe-5cf7fa0be18f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:41.374008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ab523-a507-46ee-96fe-5cf7fa0be18f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:41.374517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ab523-a507-46ee-96fe-5cf7fa0be18f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:41.398615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ab523-a507-46ee-96fe-5cf7fa0be18f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:41.407846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c3574-c57b-4e60-8ae5-08bbb416d404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:41.408776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c3574-c57b-4e60-8ae5-08bbb416d404 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:41.409274 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c3574-c57b-4e60-8ae5-08bbb416d404 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:41.409770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c3574-c57b-4e60-8ae5-08bbb416d404 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:41.410821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c3574-c57b-4e60-8ae5-08bbb416d404 None admin] RBAC: Adding query filter params (role.id=91f9d5c6716547c18da989bf327f6aca, scope.project.id=edc462e56b784f1eaefd9e0b56312647, user.id=d96eadfaf0ba4a5bada5ae72e66ec0f6) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:41.411395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c3574-c57b-4e60-8ae5-08bbb416d404 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:41.413756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4c3574-c57b-4e60-8ae5-08bbb416d404 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-262ab523-a507-46ee-96fe-5cf7fa0be18f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:41.418198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:41.419074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:41.420687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:41.424877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 116/222] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:41 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:41.434103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 107/223] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:41 2022] GET /identity/v3/role_assignments?user.id=d96eadfaf0ba4a5bada5ae72e66ec0f6&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=91f9d5c6716547c18da989bf327f6aca => generated 259 bytes in 316 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:41.436307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ab523-a507-46ee-96fe-5cf7fa0be18f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:41.472689 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:41.560406 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ab523-a507-46ee-96fe-5cf7fa0be18f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:41.618668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26bb1efe-e5e0-4b12-9c77-5fd5d5c038b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:41.619268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb1efe-e5e0-4b12-9c77-5fd5d5c038b3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:41.619556 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb1efe-e5e0-4b12-9c77-5fd5d5c038b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:41.619837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26bb1efe-e5e0-4b12-9c77-5fd5d5c038b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:41.620606 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb1efe-e5e0-4b12-9c77-5fd5d5c038b3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:41.620919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb1efe-e5e0-4b12-9c77-5fd5d5c038b3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:41.622377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26bb1efe-e5e0-4b12-9c77-5fd5d5c038b3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:41.632760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 117/224] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:41.639940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c3574-c57b-4e60-8ae5-08bbb416d404 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:41.675303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f68114-bfab-48b1-9f7c-d758d1ad43f6 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:41.675629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f68114-bfab-48b1-9f7c-d758d1ad43f6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:41.675870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f68114-bfab-48b1-9f7c-d758d1ad43f6 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:41.677420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 118/225] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:41.682793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805d577-c9c5-40db-aaf3-a82138b9daab None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:41.683077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805d577-c9c5-40db-aaf3-a82138b9daab None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:41.683353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d805d577-c9c5-40db-aaf3-a82138b9daab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:41.707202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d805d577-c9c5-40db-aaf3-a82138b9daab None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:41.708406 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d805d577-c9c5-40db-aaf3-a82138b9daab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:41.791395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c3574-c57b-4e60-8ae5-08bbb416d404 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:41.852668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 119/226] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 19:09:41.862860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606bcb9-35a2-4fe8-ac78-fcaf1d8bbd54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:41.863803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606bcb9-35a2-4fe8-ac78-fcaf1d8bbd54 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:41.864095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606bcb9-35a2-4fe8-ac78-fcaf1d8bbd54 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:41.864378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6606bcb9-35a2-4fe8-ac78-fcaf1d8bbd54 None admin] PATH_INFO: `/v3/users/0743416a65dd4d699a6b6695f3076fc0` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:41.869577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606bcb9-35a2-4fe8-ac78-fcaf1d8bbd54 None admin] RBAC: Authorizing `identity:get_user(user_id=0743416a65dd4d699a6b6695f3076fc0)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:41.871002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6606bcb9-35a2-4fe8-ac78-fcaf1d8bbd54 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:41.872747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 108/227] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:41 2022] GET /identity/v3/users/0743416a65dd4d699a6b6695f3076fc0 => generated 283 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:41.879042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805d577-c9c5-40db-aaf3-a82138b9daab None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:41.912821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:41.942009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606bcb9-35a2-4fe8-ac78-fcaf1d8bbd54 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:42.032643 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:42.045148 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d805d577-c9c5-40db-aaf3-a82138b9daab None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:42.098000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6606bcb9-35a2-4fe8-ac78-fcaf1d8bbd54 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:42.130316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1335fdc-9081-496d-9410-155289b3041c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:42.131113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1335fdc-9081-496d-9410-155289b3041c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.131394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1335fdc-9081-496d-9410-155289b3041c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.131671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1335fdc-9081-496d-9410-155289b3041c None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.133348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1335fdc-9081-496d-9410-155289b3041c None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:42.134830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1335fdc-9081-496d-9410-155289b3041c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:42.136542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 120/228] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:41 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:42.143254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1335fdc-9081-496d-9410-155289b3041c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:42.143670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077761f8-4d89-4ddb-a810-26bf9bbf371a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:42.144505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077761f8-4d89-4ddb-a810-26bf9bbf371a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.144791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077761f8-4d89-4ddb-a810-26bf9bbf371a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.145072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-077761f8-4d89-4ddb-a810-26bf9bbf371a None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.168916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-077761f8-4d89-4ddb-a810-26bf9bbf371a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-077761f8-4d89-4ddb-a810-26bf9bbf371a None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:42.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:42.178083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:42.178883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:42.188485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 109/229] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:09:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:42.193751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077761f8-4d89-4ddb-a810-26bf9bbf371a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:42.313861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:42.328588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1335fdc-9081-496d-9410-155289b3041c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:42.377398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-077761f8-4d89-4ddb-a810-26bf9bbf371a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:42.391802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:42.397652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c3b97-c5de-4cb8-bdb9-4cdb40a9518e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:42.398560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c3b97-c5de-4cb8-bdb9-4cdb40a9518e None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.398842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c3b97-c5de-4cb8-bdb9-4cdb40a9518e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.399114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-718c3b97-c5de-4cb8-bdb9-4cdb40a9518e None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647/users/0743416a65dd4d699a6b6695f3076fc0/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.402675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c3b97-c5de-4cb8-bdb9-4cdb40a9518e None admin] RBAC: Authorizing `identity:create_grant(project_id=edc462e56b784f1eaefd9e0b56312647, user_id=0743416a65dd4d699a6b6695f3076fc0, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:42.404180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-718c3b97-c5de-4cb8-bdb9-4cdb40a9518e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:42.427176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 121/230] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:42 2022] PUT /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647/users/0743416a65dd4d699a6b6695f3076fc0/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:42.444546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a67c81-1b96-4378-b5dc-56c90dfffdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:42.445252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a67c81-1b96-4378-b5dc-56c90dfffdfd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.445559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a67c81-1b96-4378-b5dc-56c90dfffdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.445925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a67c81-1b96-4378-b5dc-56c90dfffdfd None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.446720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a67c81-1b96-4378-b5dc-56c90dfffdfd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:42.447033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a67c81-1b96-4378-b5dc-56c90dfffdfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:42.448496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a67c81-1b96-4378-b5dc-56c90dfffdfd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:42.459125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 110/231] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:09:42 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:42.464635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c3b97-c5de-4cb8-bdb9-4cdb40a9518e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:42.605017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-718c3b97-c5de-4cb8-bdb9-4cdb40a9518e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:42.657834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa4374-1ea1-4be6-8f6f-c97e02903caa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:42.658729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa4374-1ea1-4be6-8f6f-c97e02903caa None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.659008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa4374-1ea1-4be6-8f6f-c97e02903caa None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.659284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3caa4374-1ea1-4be6-8f6f-c97e02903caa None admin] PATH_INFO: `/v3/users/0743416a65dd4d699a6b6695f3076fc0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.660522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa4374-1ea1-4be6-8f6f-c97e02903caa None admin] RBAC: Authorizing `identity:get_user(user_id=0743416a65dd4d699a6b6695f3076fc0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:42.662033 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3caa4374-1ea1-4be6-8f6f-c97e02903caa None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:42.663691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 122/232] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:42 2022] GET /identity/v3/users/0743416a65dd4d699a6b6695f3076fc0 => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:42.669343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a67c81-1b96-4378-b5dc-56c90dfffdfd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:42.780615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a67c81-1b96-4378-b5dc-56c90dfffdfd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:42.823147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea708eff-8a4e-4bd0-9224-1e1a85d2ddc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:42.824026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea708eff-8a4e-4bd0-9224-1e1a85d2ddc5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:42.824311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea708eff-8a4e-4bd0-9224-1e1a85d2ddc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:42.824589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea708eff-8a4e-4bd0-9224-1e1a85d2ddc5 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:42.825830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea708eff-8a4e-4bd0-9224-1e1a85d2ddc5 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:42.827402 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea708eff-8a4e-4bd0-9224-1e1a85d2ddc5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:42.829201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 111/233] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:42 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:42.835281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa4374-1ea1-4be6-8f6f-c97e02903caa None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:42.877884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.005592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3caa4374-1ea1-4be6-8f6f-c97e02903caa None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.054533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d58fb120-9b62-424a-9672-92eb6c7ac6a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.055799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fb120-9b62-424a-9672-92eb6c7ac6a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.056287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fb120-9b62-424a-9672-92eb6c7ac6a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.056765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d58fb120-9b62-424a-9672-92eb6c7ac6a3 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647/users/0743416a65dd4d699a6b6695f3076fc0/roles/77052d58af4f4b5d9b2d003531462e50` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.063517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fb120-9b62-424a-9672-92eb6c7ac6a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=edc462e56b784f1eaefd9e0b56312647, user_id=0743416a65dd4d699a6b6695f3076fc0, role_id=77052d58af4f4b5d9b2d003531462e50)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:43.065297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d58fb120-9b62-424a-9672-92eb6c7ac6a3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:43.089244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 123/234] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:42 2022] PUT /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647/users/0743416a65dd4d699a6b6695f3076fc0/roles/77052d58af4f4b5d9b2d003531462e50 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:43.119748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ddca0-ee20-44b2-9826-45b136c45d8f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.120072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ddca0-ee20-44b2-9826-45b136c45d8f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.120312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3ddca0-ee20-44b2-9826-45b136c45d8f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.121909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 112/235] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:43.127730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e684e-a414-4ac6-b941-2e33d33d759e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.128314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e684e-a414-4ac6-b941-2e33d33d759e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.128800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7e684e-a414-4ac6-b941-2e33d33d759e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.156589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ba7e684e-a414-4ac6-b941-2e33d33d759e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:43.158224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ba7e684e-a414-4ac6-b941-2e33d33d759e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:43.253837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 124/236] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:43.294385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ddca0-ee20-44b2-9826-45b136c45d8f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.334121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd8807-22f6-4d0d-94c7-a964fe8aa6af None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.335058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd8807-22f6-4d0d-94c7-a964fe8aa6af None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.335539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bd8807-22f6-4d0d-94c7-a964fe8aa6af None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.338445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 125/237] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:43.343717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609679b-a72a-45ee-a650-d465fdce88ad None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.344008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609679b-a72a-45ee-a650-d465fdce88ad None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.344288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f609679b-a72a-45ee-a650-d465fdce88ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.349826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.374586 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f609679b-a72a-45ee-a650-d465fdce88ad None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:43.376910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f609679b-a72a-45ee-a650-d465fdce88ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:43.380250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.438996 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3ddca0-ee20-44b2-9826-45b136c45d8f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.492854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 126/238] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:43.502312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b3068-b587-4b76-a9a6-45290c5bad3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.503433 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b3068-b587-4b76-a9a6-45290c5bad3d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.503921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b3068-b587-4b76-a9a6-45290c5bad3d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.504426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69b3068-b587-4b76-a9a6-45290c5bad3d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.515578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69b3068-b587-4b76-a9a6-45290c5bad3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-c69b3068-b587-4b76-a9a6-45290c5bad3d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:43.523585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:43.524492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:43.525294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:43.527710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 113/239] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:43 2022] GET /identity/v3/roles/reader => generated 84 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:43.530273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609679b-a72a-45ee-a650-d465fdce88ad None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.534684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b3068-b587-4b76-a9a6-45290c5bad3d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.679560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.686170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69b3068-b587-4b76-a9a6-45290c5bad3d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.705295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f609679b-a72a-45ee-a650-d465fdce88ad None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.725661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb93aab-29e4-49b7-b2bd-5b8e8de02904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.726696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb93aab-29e4-49b7-b2bd-5b8e8de02904 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.727236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb93aab-29e4-49b7-b2bd-5b8e8de02904 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.727740 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb93aab-29e4-49b7-b2bd-5b8e8de02904 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.728695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb93aab-29e4-49b7-b2bd-5b8e8de02904 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:43.729207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb93aab-29e4-49b7-b2bd-5b8e8de02904 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:43.731004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb93aab-29e4-49b7-b2bd-5b8e8de02904 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:43.741872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 114/240] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:43 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:43.742014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440747a6-c71b-4ad5-8f7a-61c26e3cbf73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.742850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440747a6-c71b-4ad5-8f7a-61c26e3cbf73 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.743137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440747a6-c71b-4ad5-8f7a-61c26e3cbf73 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.743443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440747a6-c71b-4ad5-8f7a-61c26e3cbf73 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:43.744693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440747a6-c71b-4ad5-8f7a-61c26e3cbf73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:43.746435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440747a6-c71b-4ad5-8f7a-61c26e3cbf73 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:43.749577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb93aab-29e4-49b7-b2bd-5b8e8de02904 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.750867 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 127/241] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:43.754705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440747a6-c71b-4ad5-8f7a-61c26e3cbf73 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:43.774358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:43.946179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb93aab-29e4-49b7-b2bd-5b8e8de02904 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.964389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440747a6-c71b-4ad5-8f7a-61c26e3cbf73 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:43.985682 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bc9434-3e13-456e-88ff-17de29ddee16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:43.987016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9434-3e13-456e-88ff-17de29ddee16 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:43.987590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9434-3e13-456e-88ff-17de29ddee16 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:43.988069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bc9434-3e13-456e-88ff-17de29ddee16 None admin] PATH_INFO: `/v3/users/7e31da272e4649e587d72c07ddb75128` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.004363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc9434-3e13-456e-88ff-17de29ddee16 None admin] RBAC: Authorizing `identity:get_user(user_id=7e31da272e4649e587d72c07ddb75128)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.006146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bc9434-3e13-456e-88ff-17de29ddee16 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.008053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 115/242] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:43 2022] GET /identity/v3/users/7e31da272e4649e587d72c07ddb75128 => generated 297 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:44.008289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc2291b-ee78-4aed-b26b-82e98e3c856d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.008896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2291b-ee78-4aed-b26b-82e98e3c856d None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.009178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2291b-ee78-4aed-b26b-82e98e3c856d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.009450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc2291b-ee78-4aed-b26b-82e98e3c856d None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.010206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2291b-ee78-4aed-b26b-82e98e3c856d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.010696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f424082-85f2-4486-9094-c9eecaec2de9 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.011181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f424082-85f2-4486-9094-c9eecaec2de9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.011620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc2291b-ee78-4aed-b26b-82e98e3c856d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.011873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f424082-85f2-4486-9094-c9eecaec2de9 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.013813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 116/243] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:44.014833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f424082-85f2-4486-9094-c9eecaec2de9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:44.039864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 128/244] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:09:43 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:44.041538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:44.043059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db2d37-656a-42fc-9130-a8c6eb706411 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.043615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db2d37-656a-42fc-9130-a8c6eb706411 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.044150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87db2d37-656a-42fc-9130-a8c6eb706411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.066665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-87db2d37-656a-42fc-9130-a8c6eb706411 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:44.068460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-87db2d37-656a-42fc-9130-a8c6eb706411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:44.134569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f424082-85f2-4486-9094-c9eecaec2de9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.141919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:44.157897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 129/245] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:44.192262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3696bc-f6ce-4ed1-9f78-f4a30a5a2586 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.192947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3696bc-f6ce-4ed1-9f78-f4a30a5a2586 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.193219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3696bc-f6ce-4ed1-9f78-f4a30a5a2586 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.193491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3696bc-f6ce-4ed1-9f78-f4a30a5a2586 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.228852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db2d37-656a-42fc-9130-a8c6eb706411 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:44.281761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3696bc-f6ce-4ed1-9f78-f4a30a5a2586 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.281761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3696bc-f6ce-4ed1-9f78-f4a30a5a2586 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.303989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:44.376977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87db2d37-656a-42fc-9130-a8c6eb706411 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.403890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 117/246] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:44.409728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3696bc-f6ce-4ed1-9f78-f4a30a5a2586 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:44.424247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823c6b7-6169-4593-b5b0-ab32473afc73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.425480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823c6b7-6169-4593-b5b0-ab32473afc73 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.425991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823c6b7-6169-4593-b5b0-ab32473afc73 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.426489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9823c6b7-6169-4593-b5b0-ab32473afc73 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.439525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9823c6b7-6169-4593-b5b0-ab32473afc73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-9823c6b7-6169-4593-b5b0-ab32473afc73 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:44.448450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:44.449348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:44.450223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:44.453392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 130/247] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:44.459246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823c6b7-6169-4593-b5b0-ab32473afc73 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:44.519624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:44.526220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3696bc-f6ce-4ed1-9f78-f4a30a5a2586 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.545831 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:44.564076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb18df-4092-4391-bf5e-995c537552f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.564999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb18df-4092-4391-bf5e-995c537552f2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.565283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb18df-4092-4391-bf5e-995c537552f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.565577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46cb18df-4092-4391-bf5e-995c537552f2 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.566984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb18df-4092-4391-bf5e-995c537552f2 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.568282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46cb18df-4092-4391-bf5e-995c537552f2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.571341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 118/248] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:44 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:44.577236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb18df-4092-4391-bf5e-995c537552f2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:44.582212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9823c6b7-6169-4593-b5b0-ab32473afc73 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.619913 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153c1c5-92d1-43bb-81e7-d224b73d6408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.620936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153c1c5-92d1-43bb-81e7-d224b73d6408 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.621445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153c1c5-92d1-43bb-81e7-d224b73d6408 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.622032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153c1c5-92d1-43bb-81e7-d224b73d6408 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.623556 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153c1c5-92d1-43bb-81e7-d224b73d6408 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:44.624142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153c1c5-92d1-43bb-81e7-d224b73d6408 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.625810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7153c1c5-92d1-43bb-81e7-d224b73d6408 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.636327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 131/249] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:44.644704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153c1c5-92d1-43bb-81e7-d224b73d6408 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:44.695104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46cb18df-4092-4391-bf5e-995c537552f2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.731886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4a8a1-1639-4d53-8d42-ad6472adb2da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.733244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4a8a1-1639-4d53-8d42-ad6472adb2da None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.733600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4a8a1-1639-4d53-8d42-ad6472adb2da None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.734011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a4a8a1-1639-4d53-8d42-ad6472adb2da None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.734851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4a8a1-1639-4d53-8d42-ad6472adb2da None admin] RBAC: Adding query filter params (role.id=997417d78f914e27958b852313dcae0e, scope.project.id=edc462e56b784f1eaefd9e0b56312647, user.id=7e31da272e4649e587d72c07ddb75128) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:44.735457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4a8a1-1639-4d53-8d42-ad6472adb2da None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.736603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a4a8a1-1639-4d53-8d42-ad6472adb2da None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.742746 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 119/250] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:44 2022] GET /identity/v3/role_assignments?user.id=7e31da272e4649e587d72c07ddb75128&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=997417d78f914e27958b852313dcae0e => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:44.762540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7153c1c5-92d1-43bb-81e7-d224b73d6408 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.780280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:44.813454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf2294-071f-4a2c-9d8c-e1c5b4ffbcf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:44.814574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf2294-071f-4a2c-9d8c-e1c5b4ffbcf8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.814963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf2294-071f-4a2c-9d8c-e1c5b4ffbcf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.815345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf2294-071f-4a2c-9d8c-e1c5b4ffbcf8 None admin] PATH_INFO: `/v3/users/d96eadfaf0ba4a5bada5ae72e66ec0f6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.818390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf2294-071f-4a2c-9d8c-e1c5b4ffbcf8 None admin] RBAC: Authorizing `identity:get_user(user_id=d96eadfaf0ba4a5bada5ae72e66ec0f6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:44.820097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf2294-071f-4a2c-9d8c-e1c5b4ffbcf8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:44.822214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 132/251] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:44 2022] GET /identity/v3/users/d96eadfaf0ba4a5bada5ae72e66ec0f6 => generated 297 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:44.828587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4a8a1-1639-4d53-8d42-ad6472adb2da None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:44.898169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee477d-60e1-47ae-836c-003d43a0f968 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.898460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee477d-60e1-47ae-836c-003d43a0f968 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.898744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10ee477d-60e1-47ae-836c-003d43a0f968 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.900316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 133/252] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:44.900432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:44.909042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acbf0a-72a8-4592-95f1-bb34e2fd7f4f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:44.909335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acbf0a-72a8-4592-95f1-bb34e2fd7f4f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:44.909757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4acbf0a-72a8-4592-95f1-bb34e2fd7f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:44.931565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e4acbf0a-72a8-4592-95f1-bb34e2fd7f4f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:44.932995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e4acbf0a-72a8-4592-95f1-bb34e2fd7f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:44.961353 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a4a8a1-1639-4d53-8d42-ad6472adb2da None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:44.999289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb7951-fff4-47b0-b012-15a1dd5e19a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:45.000169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb7951-fff4-47b0-b012-15a1dd5e19a0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:45.000449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb7951-fff4-47b0-b012-15a1dd5e19a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:45.000726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ecb7951-fff4-47b0-b012-15a1dd5e19a0 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:45.001988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb7951-fff4-47b0-b012-15a1dd5e19a0 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:45.003529 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ecb7951-fff4-47b0-b012-15a1dd5e19a0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:45.005708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 120/253] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:44 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:45.012152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb7951-fff4-47b0-b012-15a1dd5e19a0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:45.062947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 134/254] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:45.127496 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acbf0a-72a8-4592-95f1-bb34e2fd7f4f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:45.168233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ecb7951-fff4-47b0-b012-15a1dd5e19a0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:45.193976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:45.229593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa52f9-9a4a-4251-a4c0-e1cc6eae35e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:45.230506 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa52f9-9a4a-4251-a4c0-e1cc6eae35e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:45.230791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa52f9-9a4a-4251-a4c0-e1cc6eae35e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:45.231070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fa52f9-9a4a-4251-a4c0-e1cc6eae35e1 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647/users/d96eadfaf0ba4a5bada5ae72e66ec0f6/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:45.233335 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa52f9-9a4a-4251-a4c0-e1cc6eae35e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=edc462e56b784f1eaefd9e0b56312647, user_id=d96eadfaf0ba4a5bada5ae72e66ec0f6, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:45.234829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fa52f9-9a4a-4251-a4c0-e1cc6eae35e1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:45.265236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 121/255] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:45 2022] PUT /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647/users/d96eadfaf0ba4a5bada5ae72e66ec0f6/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:45.343380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4acbf0a-72a8-4592-95f1-bb34e2fd7f4f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:45.359408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:45.391653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201c0bf-ce13-4910-9442-c12f9f9531fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:45.392461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201c0bf-ce13-4910-9442-c12f9f9531fd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:45.392746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201c0bf-ce13-4910-9442-c12f9f9531fd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:45.393023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b201c0bf-ce13-4910-9442-c12f9f9531fd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:45.410558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b201c0bf-ce13-4910-9442-c12f9f9531fd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b201c0bf-ce13-4910-9442-c12f9f9531fd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:45.418573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:45.419491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:45.421829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 135/256] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:09:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:09:45.431555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa52f9-9a4a-4251-a4c0-e1cc6eae35e1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:45.557241 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fa52f9-9a4a-4251-a4c0-e1cc6eae35e1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:45.596280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1673-8b19-41f4-843d-74166c0f3f17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:45.596883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1673-8b19-41f4-843d-74166c0f3f17 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:45.597167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1673-8b19-41f4-843d-74166c0f3f17 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:45.597444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201a1673-8b19-41f4-843d-74166c0f3f17 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:45.598240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1673-8b19-41f4-843d-74166c0f3f17 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:45.598894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1673-8b19-41f4-843d-74166c0f3f17 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:45.600334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-201a1673-8b19-41f4-843d-74166c0f3f17 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:45.610962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 122/257] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:45.617542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201c0bf-ce13-4910-9442-c12f9f9531fd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:45.751899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b201c0bf-ce13-4910-9442-c12f9f9531fd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:45.799135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f65e7b-6aec-4fdc-a9b2-9d4085095d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:45.800396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f65e7b-6aec-4fdc-a9b2-9d4085095d58 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:45.800879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f65e7b-6aec-4fdc-a9b2-9d4085095d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:45.801349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f65e7b-6aec-4fdc-a9b2-9d4085095d58 None admin] PATH_INFO: `/v3/users/0743416a65dd4d699a6b6695f3076fc0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:45.802792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f65e7b-6aec-4fdc-a9b2-9d4085095d58 None admin] RBAC: Authorizing `identity:get_user(user_id=0743416a65dd4d699a6b6695f3076fc0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:45.804477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f65e7b-6aec-4fdc-a9b2-9d4085095d58 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:45.806631 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 136/258] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:45 2022] GET /identity/v3/users/0743416a65dd4d699a6b6695f3076fc0 => generated 283 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:45.812923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1673-8b19-41f4-843d-74166c0f3f17 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:45.878186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:45.889141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158ee26-e976-4e34-8f27-6dca0ce6946a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:45.889934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158ee26-e976-4e34-8f27-6dca0ce6946a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:45.890443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c158ee26-e976-4e34-8f27-6dca0ce6946a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:45.892263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 137/259] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:45 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:45.897743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b42316-1996-41da-904f-8c0f09db3041 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:45.898274 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b42316-1996-41da-904f-8c0f09db3041 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:45.898755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b42316-1996-41da-904f-8c0f09db3041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:45.934557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c2b42316-1996-41da-904f-8c0f09db3041 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:45.936885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c2b42316-1996-41da-904f-8c0f09db3041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:45.965389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201a1673-8b19-41f4-843d-74166c0f3f17 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:45.993902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:46.017257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 138/260] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:46.027171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451fdb7-9c4f-42aa-83ed-a4a7f1a46a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.027841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451fdb7-9c4f-42aa-83ed-a4a7f1a46a66 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.028133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451fdb7-9c4f-42aa-83ed-a4a7f1a46a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.028415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9451fdb7-9c4f-42aa-83ed-a4a7f1a46a66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.046752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b42316-1996-41da-904f-8c0f09db3041 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.150108 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451fdb7-9c4f-42aa-83ed-a4a7f1a46a66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:46.151564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9451fdb7-9c4f-42aa-83ed-a4a7f1a46a66 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:46.167231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b42316-1996-41da-904f-8c0f09db3041 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.207575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad6f78-c034-457a-8512-1c339387ca2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.208728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad6f78-c034-457a-8512-1c339387ca2f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.209232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad6f78-c034-457a-8512-1c339387ca2f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.209748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bad6f78-c034-457a-8512-1c339387ca2f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.222437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bad6f78-c034-457a-8512-1c339387ca2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:46.231517 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-9bad6f78-c034-457a-8512-1c339387ca2f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:46.233054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:46.234082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:46.234950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:46.238013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 139/261] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:46.244824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad6f78-c034-457a-8512-1c339387ca2f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.283276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:46.362874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 123/262] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:09:46.374016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bad6f78-c034-457a-8512-1c339387ca2f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.383605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451fdb7-9c4f-42aa-83ed-a4a7f1a46a66 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.412277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc69d6-9c89-40fa-84e3-e26bd38f88bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.413627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc69d6-9c89-40fa-84e3-e26bd38f88bd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.414112 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc69d6-9c89-40fa-84e3-e26bd38f88bd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.414585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9fc69d6-9c89-40fa-84e3-e26bd38f88bd None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.415679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc69d6-9c89-40fa-84e3-e26bd38f88bd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:46.416284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc69d6-9c89-40fa-84e3-e26bd38f88bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:46.417890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9fc69d6-9c89-40fa-84e3-e26bd38f88bd None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:46.428025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 140/263] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:46.434002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc69d6-9c89-40fa-84e3-e26bd38f88bd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.557910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:46.571895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9fc69d6-9c89-40fa-84e3-e26bd38f88bd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.598940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9451fdb7-9c4f-42aa-83ed-a4a7f1a46a66 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.635114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2c165-7cb4-4b53-8c6a-e8cc558d8ba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.635997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c165-7cb4-4b53-8c6a-e8cc558d8ba0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.636280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c165-7cb4-4b53-8c6a-e8cc558d8ba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.636557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba2c165-7cb4-4b53-8c6a-e8cc558d8ba0 None admin] PATH_INFO: `/v3/users/0743416a65dd4d699a6b6695f3076fc0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.639308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2c165-7cb4-4b53-8c6a-e8cc558d8ba0 None admin] RBAC: Authorizing `identity:get_user(user_id=0743416a65dd4d699a6b6695f3076fc0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:46.640916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba2c165-7cb4-4b53-8c6a-e8cc558d8ba0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:46.645166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 141/264] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/users/0743416a65dd4d699a6b6695f3076fc0 => generated 283 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:46.648272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2c165-7cb4-4b53-8c6a-e8cc558d8ba0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.667866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea5035-4e5c-4248-8fca-870cd9adf280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.668661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea5035-4e5c-4248-8fca-870cd9adf280 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.668946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea5035-4e5c-4248-8fca-870cd9adf280 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.669221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ea5035-4e5c-4248-8fca-870cd9adf280 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.671113 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea5035-4e5c-4248-8fca-870cd9adf280 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:46.672574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ea5035-4e5c-4248-8fca-870cd9adf280 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:46.676231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 124/265] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:46.681395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea5035-4e5c-4248-8fca-870cd9adf280 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:46.705544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:46.837338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba2c165-7cb4-4b53-8c6a-e8cc558d8ba0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.843717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ea5035-4e5c-4248-8fca-870cd9adf280 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:46.874887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95853c75-f5da-4f65-b93a-e2a32e5cfd04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.875546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95853c75-f5da-4f65-b93a-e2a32e5cfd04 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.875832 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95853c75-f5da-4f65-b93a-e2a32e5cfd04 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.876112 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95853c75-f5da-4f65-b93a-e2a32e5cfd04 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.906227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:46.921906 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ffee4-e953-44c2-ab34-1807e6c34743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:46.922805 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ffee4-e953-44c2-ab34-1807e6c34743 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:46.923096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ffee4-e953-44c2-ab34-1807e6c34743 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:46.923374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d28ffee4-e953-44c2-ab34-1807e6c34743 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:46.924209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ffee4-e953-44c2-ab34-1807e6c34743 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=edc462e56b784f1eaefd9e0b56312647, user.id=0743416a65dd4d699a6b6695f3076fc0) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:46.924515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ffee4-e953-44c2-ab34-1807e6c34743 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:46.926001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d28ffee4-e953-44c2-ab34-1807e6c34743 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:46.941571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 125/266] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/role_assignments?user.id=0743416a65dd4d699a6b6695f3076fc0&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 608 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:46.973740 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95853c75-f5da-4f65-b93a-e2a32e5cfd04 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:46.975462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95853c75-f5da-4f65-b93a-e2a32e5cfd04 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:47.117472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 142/267] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:47.123097 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ffee4-e953-44c2-ab34-1807e6c34743 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:47.238617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:47.274496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d28ffee4-e953-44c2-ab34-1807e6c34743 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:47.337263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b98de08-0663-4f11-acbe-8b89289a0099 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:47.338180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98de08-0663-4f11-acbe-8b89289a0099 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:47.338468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98de08-0663-4f11-acbe-8b89289a0099 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:47.338748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b98de08-0663-4f11-acbe-8b89289a0099 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:47.342731 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98de08-0663-4f11-acbe-8b89289a0099 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:47.344359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b98de08-0663-4f11-acbe-8b89289a0099 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:47.346455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 126/268] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:47 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:47.352683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95853c75-f5da-4f65-b93a-e2a32e5cfd04 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:47.460494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:47.527385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95853c75-f5da-4f65-b93a-e2a32e5cfd04 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:47.576660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1dcd44-be5f-4404-ae13-0d14aff9f672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:47.577716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dcd44-be5f-4404-ae13-0d14aff9f672 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:47.578227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dcd44-be5f-4404-ae13-0d14aff9f672 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:47.578727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1dcd44-be5f-4404-ae13-0d14aff9f672 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:47.579773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1dcd44-be5f-4404-ae13-0d14aff9f672 None admin] RBAC: Adding query filter params (role.id=77052d58af4f4b5d9b2d003531462e50, scope.project.id=edc462e56b784f1eaefd9e0b56312647, user.id=0743416a65dd4d699a6b6695f3076fc0) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:47.580289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1dcd44-be5f-4404-ae13-0d14aff9f672 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:47.581979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1dcd44-be5f-4404-ae13-0d14aff9f672 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:47.592851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 143/269] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:47 2022] GET /identity/v3/role_assignments?user.id=0743416a65dd4d699a6b6695f3076fc0&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=77052d58af4f4b5d9b2d003531462e50 => generated 608 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:47.782418 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b15662-1f58-4454-af4f-9214e375085a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:47.783429 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b15662-1f58-4454-af4f-9214e375085a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:47.783922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b15662-1f58-4454-af4f-9214e375085a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:47.785871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 127/270] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:47 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:47.791464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0ba32-f7f3-4028-9fb8-731432c300fc None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:47.792035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0ba32-f7f3-4028-9fb8-731432c300fc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:47.792545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e0ba32-f7f3-4028-9fb8-731432c300fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:47.815165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-68e0ba32-f7f3-4028-9fb8-731432c300fc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:47.816735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-68e0ba32-f7f3-4028-9fb8-731432c300fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:47.911460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 144/271] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:47.937069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b15662-1f58-4454-af4f-9214e375085a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:48.055728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b15662-1f58-4454-af4f-9214e375085a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:48.074203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6dca-7252-4fee-b970-8f23ed659b0c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.074843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6dca-7252-4fee-b970-8f23ed659b0c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.075343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7e6dca-7252-4fee-b970-8f23ed659b0c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.078058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 145/272] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:48.085409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0543063-8efd-474e-a651-bd5e6bb18c52 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.085915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0543063-8efd-474e-a651-bd5e6bb18c52 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.086204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0543063-8efd-474e-a651-bd5e6bb18c52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.101113 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e5cb6fc-16b8-4c5e-a929-e6e58bea9825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:48.102366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cb6fc-16b8-4c5e-a929-e6e58bea9825 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.102883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cb6fc-16b8-4c5e-a929-e6e58bea9825 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.103379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5cb6fc-16b8-4c5e-a929-e6e58bea9825 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.113801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d0543063-8efd-474e-a651-bd5e6bb18c52 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:48.114890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:48.115000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d0543063-8efd-474e-a651-bd5e6bb18c52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:48.120422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e5cb6fc-16b8-4c5e-a929-e6e58bea9825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-3e5cb6fc-16b8-4c5e-a929-e6e58bea9825 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:48.129530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:48.130610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:48.131545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:48.133637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 128/273] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:48.135027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c170097-3621-48a3-a5e5-ea1ced107b90 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.135321 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c170097-3621-48a3-a5e5-ea1ced107b90 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.135601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c170097-3621-48a3-a5e5-ea1ced107b90 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.140627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 129/274] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:48.144219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c170097-3621-48a3-a5e5-ea1ced107b90 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:48.177220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:48.197416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 146/275] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:48.199053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b50d-5170-4519-a8f6-0abd8a1e9801 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.199479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b50d-5170-4519-a8f6-0abd8a1e9801 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.199874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c931b50d-5170-4519-a8f6-0abd8a1e9801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.222392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c931b50d-5170-4519-a8f6-0abd8a1e9801 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:48.223885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c931b50d-5170-4519-a8f6-0abd8a1e9801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:48.258676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c170097-3621-48a3-a5e5-ea1ced107b90 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:48.290504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 147/276] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:48.291403 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a7b72e9-2038-4916-89bc-2761f0b75a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:48.291857 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee482-acc6-4dae-9f4a-76fcd883770a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.292144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee482-acc6-4dae-9f4a-76fcd883770a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.292242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b72e9-2038-4916-89bc-2761f0b75a6a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.292415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d85ee482-acc6-4dae-9f4a-76fcd883770a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.292508 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b72e9-2038-4916-89bc-2761f0b75a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.292695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a7b72e9-2038-4916-89bc-2761f0b75a6a None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.293442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b72e9-2038-4916-89bc-2761f0b75a6a None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:48.297839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b72e9-2038-4916-89bc-2761f0b75a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:48.297995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 148/277] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:48.297995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ee482-acc6-4dae-9f4a-76fcd883770a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:48.298166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a7b72e9-2038-4916-89bc-2761f0b75a6a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:48.306058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 130/278] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:48.311687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59235c1c-70e3-4e5f-bd54-aee2b5e3a81a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.311978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59235c1c-70e3-4e5f-bd54-aee2b5e3a81a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.314214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59235c1c-70e3-4e5f-bd54-aee2b5e3a81a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.331915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-59235c1c-70e3-4e5f-bd54-aee2b5e3a81a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:48.333130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-59235c1c-70e3-4e5f-bd54-aee2b5e3a81a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:48.390502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:48.401553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 131/279] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:48.402375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59235c1c-70e3-4e5f-bd54-aee2b5e3a81a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:48.406280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d85ee482-acc6-4dae-9f4a-76fcd883770a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:48.420359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:48.442466 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cbb03-f0e9-4c0d-bf89-3d70024fb0fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:48.443246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cbb03-f0e9-4c0d-bf89-3d70024fb0fc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.443526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cbb03-f0e9-4c0d-bf89-3d70024fb0fc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.443791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa6cbb03-f0e9-4c0d-bf89-3d70024fb0fc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.457237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa6cbb03-f0e9-4c0d-bf89-3d70024fb0fc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-aa6cbb03-f0e9-4c0d-bf89-3d70024fb0fc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:48.464822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:48.466187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 149/280] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:48 2022] GET /identity/v3/roles/reader => generated 84 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:48.467462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cbb03-f0e9-4c0d-bf89-3d70024fb0fc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:48.514843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59235c1c-70e3-4e5f-bd54-aee2b5e3a81a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:48.548188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd90ab4-238c-4f58-8521-e9304a843bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:48.549064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd90ab4-238c-4f58-8521-e9304a843bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.549354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd90ab4-238c-4f58-8521-e9304a843bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.549660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd90ab4-238c-4f58-8521-e9304a843bf5 None admin] PATH_INFO: `/v3/users/b0f4b7e1333c4648ba9ddb08e70c45b4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.563845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd90ab4-238c-4f58-8521-e9304a843bf5 None admin] RBAC: Authorizing `identity:get_user(user_id=b0f4b7e1333c4648ba9ddb08e70c45b4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:48.565245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd90ab4-238c-4f58-8521-e9304a843bf5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:48.566956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 132/281] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:48 2022] GET /identity/v3/users/b0f4b7e1333c4648ba9ddb08e70c45b4 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:48.571806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd90ab4-238c-4f58-8521-e9304a843bf5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:48.572296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa6cbb03-f0e9-4c0d-bf89-3d70024fb0fc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:48.606211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e98cc4-b7ba-4d35-8d5b-536b5e9a3a33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:48.607524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e98cc4-b7ba-4d35-8d5b-536b5e9a3a33 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.607886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e98cc4-b7ba-4d35-8d5b-536b5e9a3a33 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.608175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e98cc4-b7ba-4d35-8d5b-536b5e9a3a33 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.609392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e98cc4-b7ba-4d35-8d5b-536b5e9a3a33 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:48.610889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e98cc4-b7ba-4d35-8d5b-536b5e9a3a33 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:48.612533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 150/282] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:48.616893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e98cc4-b7ba-4d35-8d5b-536b5e9a3a33 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:48.621525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:48.656133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:48.671946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd90ab4-238c-4f58-8521-e9304a843bf5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:48.706126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520636c5-7d7f-4da1-bf7c-feb87616b391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:48.706919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520636c5-7d7f-4da1-bf7c-feb87616b391 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.707204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520636c5-7d7f-4da1-bf7c-feb87616b391 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.707480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520636c5-7d7f-4da1-bf7c-feb87616b391 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.713807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e98cc4-b7ba-4d35-8d5b-536b5e9a3a33 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:48.717466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520636c5-7d7f-4da1-bf7c-feb87616b391 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-520636c5-7d7f-4da1-bf7c-feb87616b391 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:48.725411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:48.726490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:48.727364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 133/283] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:48 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:48.728017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520636c5-7d7f-4da1-bf7c-feb87616b391 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:48.751502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0ef59-a501-4151-bfc2-0c9008f8f834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:48.752168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0ef59-a501-4151-bfc2-0c9008f8f834 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.752463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0ef59-a501-4151-bfc2-0c9008f8f834 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.752738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd0ef59-a501-4151-bfc2-0c9008f8f834 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.753487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0ef59-a501-4151-bfc2-0c9008f8f834 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:48.753814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0ef59-a501-4151-bfc2-0c9008f8f834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:48.755211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd0ef59-a501-4151-bfc2-0c9008f8f834 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:48.763799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 151/284] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:48 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:48.764634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0ef59-a501-4151-bfc2-0c9008f8f834 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:48.829991 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520636c5-7d7f-4da1-bf7c-feb87616b391 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:48.839839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:48.863648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd0ef59-a501-4151-bfc2-0c9008f8f834 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:48.864246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ede4f-50c8-460d-a79d-bee17c7dc6a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:48.864910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ede4f-50c8-460d-a79d-bee17c7dc6a1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.865200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ede4f-50c8-460d-a79d-bee17c7dc6a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.865481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ede4f-50c8-460d-a79d-bee17c7dc6a1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.866022 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:48.866346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ede4f-50c8-460d-a79d-bee17c7dc6a1 None admin] RBAC: Adding query filter params (role.id=91f9d5c6716547c18da989bf327f6aca, user.id=b0f4b7e1333c4648ba9ddb08e70c45b4, scope.system=all) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:48.866665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ede4f-50c8-460d-a79d-bee17c7dc6a1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:48.868126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e14ede4f-50c8-460d-a79d-bee17c7dc6a1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:48.873839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 134/285] 10.222.0.69 () {56 vars in 1308 bytes} [Wed Jul 20 19:09:48 2022] GET /identity/v3/role_assignments?user.id=b0f4b7e1333c4648ba9ddb08e70c45b4&scope.system=all&role.id=91f9d5c6716547c18da989bf327f6aca => generated 226 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:48.874645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ede4f-50c8-460d-a79d-bee17c7dc6a1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:48.908181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493dae6-8115-48db-b060-cf9c55210f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:48.909217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493dae6-8115-48db-b060-cf9c55210f6a None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:48.909672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493dae6-8115-48db-b060-cf9c55210f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:48.910106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6493dae6-8115-48db-b060-cf9c55210f6a None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:48.911094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493dae6-8115-48db-b060-cf9c55210f6a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:48.912802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6493dae6-8115-48db-b060-cf9c55210f6a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:48.938437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 152/286] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:09:48 2022] POST /identity/v3/users => generated 293 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:09:48.939692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493dae6-8115-48db-b060-cf9c55210f6a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:49.026990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e14ede4f-50c8-460d-a79d-bee17c7dc6a1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:49.075499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17115fd-13df-43fb-98db-6f624360de4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:49.076866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17115fd-13df-43fb-98db-6f624360de4a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.077303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17115fd-13df-43fb-98db-6f624360de4a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.078014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17115fd-13df-43fb-98db-6f624360de4a None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.079706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17115fd-13df-43fb-98db-6f624360de4a None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:49.080360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17115fd-13df-43fb-98db-6f624360de4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:49.082763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17115fd-13df-43fb-98db-6f624360de4a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:49.088724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6493dae6-8115-48db-b060-cf9c55210f6a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:49.100663 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 135/287] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:49.103557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17115fd-13df-43fb-98db-6f624360de4a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:49.138791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c3091-e711-40fd-ab54-24ffdeee5668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:49.139772 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c3091-e711-40fd-ab54-24ffdeee5668 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.140574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c3091-e711-40fd-ab54-24ffdeee5668 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.140870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233c3091-e711-40fd-ab54-24ffdeee5668 None admin] PATH_INFO: `/v3/users/7e31da272e4649e587d72c07ddb75128` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.142431 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c3091-e711-40fd-ab54-24ffdeee5668 None admin] RBAC: Authorizing `identity:get_user(user_id=7e31da272e4649e587d72c07ddb75128)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:49.144072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233c3091-e711-40fd-ab54-24ffdeee5668 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:49.146346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 153/288] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:48 2022] GET /identity/v3/users/7e31da272e4649e587d72c07ddb75128 => generated 297 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:49.151609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:49.152296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c3091-e711-40fd-ab54-24ffdeee5668 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:49.194348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:49.243566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17115fd-13df-43fb-98db-6f624360de4a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:49.287672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03ff71-8fda-4cd9-80a1-eb6f0b0a4297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:49.288656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03ff71-8fda-4cd9-80a1-eb6f0b0a4297 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.289010 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03ff71-8fda-4cd9-80a1-eb6f0b0a4297 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.289435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa03ff71-8fda-4cd9-80a1-eb6f0b0a4297 None admin] PATH_INFO: `/v3/users/d96eadfaf0ba4a5bada5ae72e66ec0f6` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.290668 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03ff71-8fda-4cd9-80a1-eb6f0b0a4297 None admin] RBAC: Authorizing `identity:get_user(user_id=d96eadfaf0ba4a5bada5ae72e66ec0f6)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:49.292237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa03ff71-8fda-4cd9-80a1-eb6f0b0a4297 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:49.295571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 136/289] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:49 2022] GET /identity/v3/users/d96eadfaf0ba4a5bada5ae72e66ec0f6 => generated 297 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:49.295812 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233c3091-e711-40fd-ab54-24ffdeee5668 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:49.300901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03ff71-8fda-4cd9-80a1-eb6f0b0a4297 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:49.347836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932c8e2-1965-49e0-86a4-eadfd53314bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:49.348890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932c8e2-1965-49e0-86a4-eadfd53314bb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.349166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932c8e2-1965-49e0-86a4-eadfd53314bb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.349437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5932c8e2-1965-49e0-86a4-eadfd53314bb None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.351168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932c8e2-1965-49e0-86a4-eadfd53314bb None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:49.352636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5932c8e2-1965-49e0-86a4-eadfd53314bb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:49.354731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 154/290] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:49 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:49.362769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932c8e2-1965-49e0-86a4-eadfd53314bb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:49.449434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa03ff71-8fda-4cd9-80a1-eb6f0b0a4297 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:49.452476 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:49.471088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:49.477601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5932c8e2-1965-49e0-86a4-eadfd53314bb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:49.495809 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52c5b2-3f3f-48f9-8108-19a772e35ef7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:49.496827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52c5b2-3f3f-48f9-8108-19a772e35ef7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.497256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52c5b2-3f3f-48f9-8108-19a772e35ef7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.497766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d52c5b2-3f3f-48f9-8108-19a772e35ef7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.514749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6eb7c-b6a9-4e09-870b-5bfe5607191e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:49.516047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6eb7c-b6a9-4e09-870b-5bfe5607191e None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.516812 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6eb7c-b6a9-4e09-870b-5bfe5607191e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.516812 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc6eb7c-b6a9-4e09-870b-5bfe5607191e None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647/users/7e31da272e4649e587d72c07ddb75128/roles/997417d78f914e27958b852313dcae0e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.519049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6eb7c-b6a9-4e09-870b-5bfe5607191e None admin] RBAC: Authorizing `identity:create_grant(project_id=edc462e56b784f1eaefd9e0b56312647, user_id=7e31da272e4649e587d72c07ddb75128, role_id=997417d78f914e27958b852313dcae0e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:49.520710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc6eb7c-b6a9-4e09-870b-5bfe5607191e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:49.538358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 155/291] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:09:49 2022] PUT /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647/users/7e31da272e4649e587d72c07ddb75128/roles/997417d78f914e27958b852313dcae0e => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:09:49.593057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52c5b2-3f3f-48f9-8108-19a772e35ef7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:49.594646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d52c5b2-3f3f-48f9-8108-19a772e35ef7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:49.711146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 137/292] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:09:49.718717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6eb7c-b6a9-4e09-870b-5bfe5607191e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:49.881071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc6eb7c-b6a9-4e09-870b-5bfe5607191e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:49.925281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7df0066-3727-4a7b-843f-fdf4b822b6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:49.926223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0066-3727-4a7b-843f-fdf4b822b6fb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:49.926502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0066-3727-4a7b-843f-fdf4b822b6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:49.926776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7df0066-3727-4a7b-843f-fdf4b822b6fb None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:49.928561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df0066-3727-4a7b-843f-fdf4b822b6fb None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:49.930105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7df0066-3727-4a7b-843f-fdf4b822b6fb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:49.931947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 156/293] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:49 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:49.938510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52c5b2-3f3f-48f9-8108-19a772e35ef7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:49.941734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:50.059627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d52c5b2-3f3f-48f9-8108-19a772e35ef7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:50.109729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbf851f-76f7-482b-93ab-c6c912fbdf3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:50.110410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf851f-76f7-482b-93ab-c6c912fbdf3a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:50.110690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf851f-76f7-482b-93ab-c6c912fbdf3a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:50.110969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbf851f-76f7-482b-93ab-c6c912fbdf3a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:50.111809 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf851f-76f7-482b-93ab-c6c912fbdf3a None admin] RBAC: Adding query filter params (role.id=91f9d5c6716547c18da989bf327f6aca, scope.project.id=edc462e56b784f1eaefd9e0b56312647, user.id=d96eadfaf0ba4a5bada5ae72e66ec0f6) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:50.112117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf851f-76f7-482b-93ab-c6c912fbdf3a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:50.113579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbf851f-76f7-482b-93ab-c6c912fbdf3a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:50.127436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 138/294] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:49 2022] GET /identity/v3/role_assignments?user.id=d96eadfaf0ba4a5bada5ae72e66ec0f6&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=91f9d5c6716547c18da989bf327f6aca => generated 608 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:51.057816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05719d4-d852-482a-8405-9a4719214116 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:51.058614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05719d4-d852-482a-8405-9a4719214116 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:51.058614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e05719d4-d852-482a-8405-9a4719214116 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:51.059970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 157/295] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:51.065422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e8288-5a66-4c3e-aaee-b158d950c809 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:51.066112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e8288-5a66-4c3e-aaee-b158d950c809 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:51.066112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e8288-5a66-4c3e-aaee-b158d950c809 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:51.099071 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-013e8288-5a66-4c3e-aaee-b158d950c809 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:51.101492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-013e8288-5a66-4c3e-aaee-b158d950c809 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:51.216944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:51.227923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 139/296] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:51.250426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05719d4-d852-482a-8405-9a4719214116 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:51.276791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:51.379124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e05719d4-d852-482a-8405-9a4719214116 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:51.417290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702957a-3e91-4a7f-9037-e3b4dfaac0e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:51.418545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702957a-3e91-4a7f-9037-e3b4dfaac0e5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:51.419030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702957a-3e91-4a7f-9037-e3b4dfaac0e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:51.419575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b702957a-3e91-4a7f-9037-e3b4dfaac0e5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:51.431135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b702957a-3e91-4a7f-9037-e3b4dfaac0e5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b702957a-3e91-4a7f-9037-e3b4dfaac0e5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:51.440276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:51.441212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:51.442055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:51.444906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 158/297] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:51.453580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e8288-5a66-4c3e-aaee-b158d950c809 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:51.555726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e8288-5a66-4c3e-aaee-b158d950c809 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:51.592962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd0bcc6-c9b3-4e8c-bb6c-6a41d9031287 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:51.594033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0bcc6-c9b3-4e8c-bb6c-6a41d9031287 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:51.594522 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0bcc6-c9b3-4e8c-bb6c-6a41d9031287 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:51.594997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd0bcc6-c9b3-4e8c-bb6c-6a41d9031287 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:51.595954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0bcc6-c9b3-4e8c-bb6c-6a41d9031287 None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:51.596480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0bcc6-c9b3-4e8c-bb6c-6a41d9031287 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:51.598147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd0bcc6-c9b3-4e8c-bb6c-6a41d9031287 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:51.608108 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 140/298] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:51.614592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702957a-3e91-4a7f-9037-e3b4dfaac0e5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:51.706848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:51.729350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b702957a-3e91-4a7f-9037-e3b4dfaac0e5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:51.746782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37fb1a-30da-4926-8d3d-61b1a77f2a45 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:51.747460 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37fb1a-30da-4926-8d3d-61b1a77f2a45 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:51.748122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb37fb1a-30da-4926-8d3d-61b1a77f2a45 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:51.750465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 141/299] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:51.756721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fd9ad-70df-42b8-8c15-54ad3a4b1ff7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:51.757247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fd9ad-70df-42b8-8c15-54ad3a4b1ff7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:51.757889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9fd9ad-70df-42b8-8c15-54ad3a4b1ff7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:51.770322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c29f0-da31-4b20-b5eb-f771bf2f937b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:51.771192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c29f0-da31-4b20-b5eb-f771bf2f937b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:51.771471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c29f0-da31-4b20-b5eb-f771bf2f937b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:51.771740 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6c29f0-da31-4b20-b5eb-f771bf2f937b None admin] PATH_INFO: `/v3/users/b0f4b7e1333c4648ba9ddb08e70c45b4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:51.772941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c29f0-da31-4b20-b5eb-f771bf2f937b None admin] RBAC: Authorizing `identity:get_user(user_id=b0f4b7e1333c4648ba9ddb08e70c45b4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:51.774415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6c29f0-da31-4b20-b5eb-f771bf2f937b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:51.776330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 159/300] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:51 2022] GET /identity/v3/users/b0f4b7e1333c4648ba9ddb08e70c45b4 => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:51.780974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ab9fd9ad-70df-42b8-8c15-54ad3a4b1ff7 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:51.782572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ab9fd9ad-70df-42b8-8c15-54ad3a4b1ff7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:51.785367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c29f0-da31-4b20-b5eb-f771bf2f937b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:51.854206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:51.869591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 142/301] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:51.891900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fd9ad-70df-42b8-8c15-54ad3a4b1ff7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:51.898855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6c29f0-da31-4b20-b5eb-f771bf2f937b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:51.940297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcadec0f-61d7-421e-97dd-355ab598a75d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:51.941475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadec0f-61d7-421e-97dd-355ab598a75d None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:51.941784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadec0f-61d7-421e-97dd-355ab598a75d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:51.942218 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcadec0f-61d7-421e-97dd-355ab598a75d None admin] PATH_INFO: `/v3/system/users/b0f4b7e1333c4648ba9ddb08e70c45b4/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:51.944008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadec0f-61d7-421e-97dd-355ab598a75d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b0f4b7e1333c4648ba9ddb08e70c45b4, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:51.945435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcadec0f-61d7-421e-97dd-355ab598a75d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:51.961549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 160/302] 10.222.0.69 () {58 vars in 1334 bytes} [Wed Jul 20 19:09:51 2022] PUT /identity/v3/system/users/b0f4b7e1333c4648ba9ddb08e70c45b4/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:51.961549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e2e6-e7d4-45b4-b067-9ede47ec25dd None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:51.961549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e2e6-e7d4-45b4-b067-9ede47ec25dd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:51.961549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a60e2e6-e7d4-45b4-b067-9ede47ec25dd None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:51.961549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 161/303] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:51.968703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9a358-86c1-442b-a1b1-376e51efbb12 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:51.969064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9a358-86c1-442b-a1b1-376e51efbb12 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:51.969259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d9a358-86c1-442b-a1b1-376e51efbb12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:51.992216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c7d9a358-86c1-442b-a1b1-376e51efbb12 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:51.994172 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c7d9a358-86c1-442b-a1b1-376e51efbb12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:52.002301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9fd9ad-70df-42b8-8c15-54ad3a4b1ff7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:52.009716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:52.056101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dac9c-fdfd-4b2e-876b-9a784a742397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:52.056925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dac9c-fdfd-4b2e-876b-9a784a742397 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.057212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dac9c-fdfd-4b2e-876b-9a784a742397 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.057494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038dac9c-fdfd-4b2e-876b-9a784a742397 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.058789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dac9c-fdfd-4b2e-876b-9a784a742397 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:52.060490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038dac9c-fdfd-4b2e-876b-9a784a742397 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:52.062588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 143/304] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:52.068336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dac9c-fdfd-4b2e-876b-9a784a742397 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:52.105806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 162/305] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:52.133087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9a358-86c1-442b-a1b1-376e51efbb12 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:52.139188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:52.195586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038dac9c-fdfd-4b2e-876b-9a784a742397 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:52.241284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92a038-8aef-4b3b-aaf8-aadeb2eb38cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:52.242044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92a038-8aef-4b3b-aaf8-aadeb2eb38cf None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.242565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92a038-8aef-4b3b-aaf8-aadeb2eb38cf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.242898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf92a038-8aef-4b3b-aaf8-aadeb2eb38cf None admin] PATH_INFO: `/v3/groups` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.243653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92a038-8aef-4b3b-aaf8-aadeb2eb38cf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:52.245988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf92a038-8aef-4b3b-aaf8-aadeb2eb38cf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:52.253737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d9a358-86c1-442b-a1b1-376e51efbb12 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:52.266727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 144/306] 10.222.0.69 () {60 vars in 1134 bytes} [Wed Jul 20 19:09:52 2022] POST /identity/v3/groups => generated 229 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:09:52.300026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4c1a5d-f0db-4243-9926-3a8edf6d0bf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:52.300920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c1a5d-f0db-4243-9926-3a8edf6d0bf0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.301191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c1a5d-f0db-4243-9926-3a8edf6d0bf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.301464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4c1a5d-f0db-4243-9926-3a8edf6d0bf0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.325597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4c1a5d-f0db-4243-9926-3a8edf6d0bf0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2c4c1a5d-f0db-4243-9926-3a8edf6d0bf0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:52.334203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:52.335259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:52.336114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 163/307] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:52.344296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92a038-8aef-4b3b-aaf8-aadeb2eb38cf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:52.450399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf257d-cbba-4fcf-8c88-9ed60a28cdc5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.450745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf257d-cbba-4fcf-8c88-9ed60a28cdc5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.450991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbf257d-cbba-4fcf-8c88-9ed60a28cdc5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.452703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 164/308] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:52.458038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f20b3-32ea-4fab-8cbb-c31925236718 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.458604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f20b3-32ea-4fab-8cbb-c31925236718 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.459097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833f20b3-32ea-4fab-8cbb-c31925236718 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.465500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf92a038-8aef-4b3b-aaf8-aadeb2eb38cf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:52.477886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:52.481114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-833f20b3-32ea-4fab-8cbb-c31925236718 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:52.484268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-833f20b3-32ea-4fab-8cbb-c31925236718 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:52.488330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:52.509775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898339f-a7e3-440d-9fcd-7e5b66738779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:52.510456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898339f-a7e3-440d-9fcd-7e5b66738779 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.510811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898339f-a7e3-440d-9fcd-7e5b66738779 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.511411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b898339f-a7e3-440d-9fcd-7e5b66738779 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.511766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898339f-a7e3-440d-9fcd-7e5b66738779 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:52.512072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898339f-a7e3-440d-9fcd-7e5b66738779 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:52.514882 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b898339f-a7e3-440d-9fcd-7e5b66738779 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:52.523261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 145/309] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:52.528762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898339f-a7e3-440d-9fcd-7e5b66738779 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:52.587760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 165/310] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:52.613025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f20b3-32ea-4fab-8cbb-c31925236718 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:52.648490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b898339f-a7e3-440d-9fcd-7e5b66738779 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:52.684814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c0c28-4c65-43bd-8f22-fbcfa9ada7a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:52.686406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c0c28-4c65-43bd-8f22-fbcfa9ada7a0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.686689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c0c28-4c65-43bd-8f22-fbcfa9ada7a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.686962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517c0c28-4c65-43bd-8f22-fbcfa9ada7a0 None admin] PATH_INFO: `/v3/users/83a6d5a5d95d4c6d969e10a3cd14f78d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.702338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c0c28-4c65-43bd-8f22-fbcfa9ada7a0 None admin] RBAC: Authorizing `identity:get_user(user_id=83a6d5a5d95d4c6d969e10a3cd14f78d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:52.703838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517c0c28-4c65-43bd-8f22-fbcfa9ada7a0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:52.705939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 146/311] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:52 2022] GET /identity/v3/users/83a6d5a5d95d4c6d969e10a3cd14f78d => generated 293 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:52.711538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c0c28-4c65-43bd-8f22-fbcfa9ada7a0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:52.726702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833f20b3-32ea-4fab-8cbb-c31925236718 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:52.755911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:52.767761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3b5e1-b9d2-4209-83a4-8f5c7b07422d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:52.768921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3b5e1-b9d2-4209-83a4-8f5c7b07422d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.769406 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3b5e1-b9d2-4209-83a4-8f5c7b07422d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.769925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3b5e1-b9d2-4209-83a4-8f5c7b07422d None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.773040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:52.783215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3b5e1-b9d2-4209-83a4-8f5c7b07422d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-35a3b5e1-b9d2-4209-83a4-8f5c7b07422d None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:52.791890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:52.792788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:52.794011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:52.801474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 166/312] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:52 2022] GET /identity/v3/roles/reader => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:52.802990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3b5e1-b9d2-4209-83a4-8f5c7b07422d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:52.821359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517c0c28-4c65-43bd-8f22-fbcfa9ada7a0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:52.859865 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e8459-ea8b-41fd-9a00-593644dd2f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:52.860531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e8459-ea8b-41fd-9a00-593644dd2f68 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.860811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e8459-ea8b-41fd-9a00-593644dd2f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.861083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e8459-ea8b-41fd-9a00-593644dd2f68 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.861938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e8459-ea8b-41fd-9a00-593644dd2f68 None admin] RBAC: Adding query filter params (role.id=997417d78f914e27958b852313dcae0e, user.id=83a6d5a5d95d4c6d969e10a3cd14f78d, scope.system=all) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:52.862245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e8459-ea8b-41fd-9a00-593644dd2f68 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:52.863717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e8459-ea8b-41fd-9a00-593644dd2f68 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:52.877872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 147/313] 10.222.0.69 () {56 vars in 1308 bytes} [Wed Jul 20 19:09:52 2022] GET /identity/v3/role_assignments?user.id=83a6d5a5d95d4c6d969e10a3cd14f78d&scope.system=all&role.id=997417d78f914e27958b852313dcae0e => generated 226 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:52.918841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3b5e1-b9d2-4209-83a4-8f5c7b07422d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:52.958962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707d14c-8f4b-4705-bec5-d6a7ea02c365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:52.959997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707d14c-8f4b-4705-bec5-d6a7ea02c365 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:52.960486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707d14c-8f4b-4705-bec5-d6a7ea02c365 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:52.960963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4707d14c-8f4b-4705-bec5-d6a7ea02c365 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:52.961975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707d14c-8f4b-4705-bec5-d6a7ea02c365 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:52.962549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707d14c-8f4b-4705-bec5-d6a7ea02c365 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:52.964588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4707d14c-8f4b-4705-bec5-d6a7ea02c365 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:52.979864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 167/314] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:52 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:52.981609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e8459-ea8b-41fd-9a00-593644dd2f68 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.095317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e8459-ea8b-41fd-9a00-593644dd2f68 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.096860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:53.132254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f712481-3cf9-43bd-a73f-feb527ba5baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.133126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f712481-3cf9-43bd-a73f-feb527ba5baf None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.133407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f712481-3cf9-43bd-a73f-feb527ba5baf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.133708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f712481-3cf9-43bd-a73f-feb527ba5baf None admin] PATH_INFO: `/v3/users/7e31da272e4649e587d72c07ddb75128` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.134909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f712481-3cf9-43bd-a73f-feb527ba5baf None admin] RBAC: Authorizing `identity:get_user(user_id=7e31da272e4649e587d72c07ddb75128)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:53.136370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f712481-3cf9-43bd-a73f-feb527ba5baf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:53.138112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 148/315] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:52 2022] GET /identity/v3/users/7e31da272e4649e587d72c07ddb75128 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:53.144032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707d14c-8f4b-4705-bec5-d6a7ea02c365 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.210240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:53.253904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4707d14c-8f4b-4705-bec5-d6a7ea02c365 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.288472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d031431-6ce7-4a5f-88cf-61b2a6823cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.289130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d031431-6ce7-4a5f-88cf-61b2a6823cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.289416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d031431-6ce7-4a5f-88cf-61b2a6823cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.289723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d031431-6ce7-4a5f-88cf-61b2a6823cf5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.362522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d031431-6ce7-4a5f-88cf-61b2a6823cf5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:53.364037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d031431-6ce7-4a5f-88cf-61b2a6823cf5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:53.426021 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:53.473042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 168/316] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:53.474670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f712481-3cf9-43bd-a73f-feb527ba5baf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.583383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f712481-3cf9-43bd-a73f-feb527ba5baf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.618137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a107e99d-a702-418c-b74e-f87aea285e39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.619003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107e99d-a702-418c-b74e-f87aea285e39 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.619289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107e99d-a702-418c-b74e-f87aea285e39 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.619567 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a107e99d-a702-418c-b74e-f87aea285e39 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.620775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107e99d-a702-418c-b74e-f87aea285e39 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:53.622304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a107e99d-a702-418c-b74e-f87aea285e39 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:53.629722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 149/317] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:53.631158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d031431-6ce7-4a5f-88cf-61b2a6823cf5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:53.736558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d031431-6ce7-4a5f-88cf-61b2a6823cf5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:53.772595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93864d8-1aad-40c0-962a-b50c9eba0ec4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:53.773264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93864d8-1aad-40c0-962a-b50c9eba0ec4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:53.773582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93864d8-1aad-40c0-962a-b50c9eba0ec4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:53.773865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93864d8-1aad-40c0-962a-b50c9eba0ec4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:53.774698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93864d8-1aad-40c0-962a-b50c9eba0ec4 None admin] RBAC: Adding query filter params (role.id=997417d78f914e27958b852313dcae0e, scope.project.id=edc462e56b784f1eaefd9e0b56312647, user.id=7e31da272e4649e587d72c07ddb75128) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:53.775010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93864d8-1aad-40c0-962a-b50c9eba0ec4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:53.776454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93864d8-1aad-40c0-962a-b50c9eba0ec4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:53.789664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 169/318] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:09:53 2022] GET /identity/v3/role_assignments?user.id=7e31da272e4649e587d72c07ddb75128&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=997417d78f914e27958b852313dcae0e => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:54.155231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a881cb4-b395-41c1-af69-3c5bd630e540 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.156222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a881cb4-b395-41c1-af69-3c5bd630e540 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.156716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a881cb4-b395-41c1-af69-3c5bd630e540 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.158368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 150/319] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:54.164016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae43da-ca79-43ac-aae6-2f1027097647 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.164445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae43da-ca79-43ac-aae6-2f1027097647 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.164617 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae43da-ca79-43ac-aae6-2f1027097647 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.182248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9bae43da-ca79-43ac-aae6-2f1027097647 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:54.183446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9bae43da-ca79-43ac-aae6-2f1027097647 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:54.202218 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.256078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 170/320] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:54.280240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a881cb4-b395-41c1-af69-3c5bd630e540 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.331347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.395270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a881cb4-b395-41c1-af69-3c5bd630e540 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.425236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b319e3-bd2a-441c-9cd8-4b7ffaf41ae3 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.425471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b319e3-bd2a-441c-9cd8-4b7ffaf41ae3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.425753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b319e3-bd2a-441c-9cd8-4b7ffaf41ae3 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.427312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 171/321] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:54.429813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ba737-6f36-4534-bcca-dbf44dcdd858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.430615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ba737-6f36-4534-bcca-dbf44dcdd858 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.431049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ba737-6f36-4534-bcca-dbf44dcdd858 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.431151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1ba737-6f36-4534-bcca-dbf44dcdd858 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.432222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841bab4-b0be-4c49-b548-ac94548a3e74 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.432511 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841bab4-b0be-4c49-b548-ac94548a3e74 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.432795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6841bab4-b0be-4c49-b548-ac94548a3e74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.442802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1ba737-6f36-4534-bcca-dbf44dcdd858 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:54.450249 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6841bab4-b0be-4c49-b548-ac94548a3e74 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:54.452072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6841bab4-b0be-4c49-b548-ac94548a3e74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-7d1ba737-6f36-4534-bcca-dbf44dcdd858 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:54.452840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:54.453971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:54.454794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 151/322] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:54.459708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ba737-6f36-4534-bcca-dbf44dcdd858 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.525455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 172/323] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:54.548289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841bab4-b0be-4c49-b548-ac94548a3e74 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.565629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.571441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1ba737-6f36-4534-bcca-dbf44dcdd858 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.610672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7836de2-2b1b-425b-9005-3c4d6213dc83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.611458 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7836de2-2b1b-425b-9005-3c4d6213dc83 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.611743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7836de2-2b1b-425b-9005-3c4d6213dc83 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.612177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7836de2-2b1b-425b-9005-3c4d6213dc83 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.612928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7836de2-2b1b-425b-9005-3c4d6213dc83 None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:54.613265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7836de2-2b1b-425b-9005-3c4d6213dc83 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.614764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7836de2-2b1b-425b-9005-3c4d6213dc83 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.629922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 152/324] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:54.630625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7836de2-2b1b-425b-9005-3c4d6213dc83 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.633113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.652792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6841bab4-b0be-4c49-b548-ac94548a3e74 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.688393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63796c4-2cb4-48b6-9919-d8c0b4bafbdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.689094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63796c4-2cb4-48b6-9919-d8c0b4bafbdf None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.689380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63796c4-2cb4-48b6-9919-d8c0b4bafbdf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.689690 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63796c4-2cb4-48b6-9919-d8c0b4bafbdf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.690871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63796c4-2cb4-48b6-9919-d8c0b4bafbdf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.692327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63796c4-2cb4-48b6-9919-d8c0b4bafbdf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.694153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 173/325] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:54.699460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63796c4-2cb4-48b6-9919-d8c0b4bafbdf None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.752191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7836de2-2b1b-425b-9005-3c4d6213dc83 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.789220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aceb8e7-ba22-43f7-b545-5494e509a5f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.790129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aceb8e7-ba22-43f7-b545-5494e509a5f9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.790414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aceb8e7-ba22-43f7-b545-5494e509a5f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.790693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aceb8e7-ba22-43f7-b545-5494e509a5f9 None admin] PATH_INFO: `/v3/users/b0f4b7e1333c4648ba9ddb08e70c45b4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.791885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aceb8e7-ba22-43f7-b545-5494e509a5f9 None admin] RBAC: Authorizing `identity:get_user(user_id=b0f4b7e1333c4648ba9ddb08e70c45b4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.793338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aceb8e7-ba22-43f7-b545-5494e509a5f9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.796337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 153/326] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/users/b0f4b7e1333c4648ba9ddb08e70c45b4 => generated 293 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:54.801976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aceb8e7-ba22-43f7-b545-5494e509a5f9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:54.809106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63796c4-2cb4-48b6-9919-d8c0b4bafbdf None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.838437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:54.848123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e643cc-6df7-4e6e-a836-fe3f0750d326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.849265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e643cc-6df7-4e6e-a836-fe3f0750d326 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.849580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e643cc-6df7-4e6e-a836-fe3f0750d326 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.850000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e643cc-6df7-4e6e-a836-fe3f0750d326 None admin] PATH_INFO: `/v3/groups` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.850706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e643cc-6df7-4e6e-a836-fe3f0750d326 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.852209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e643cc-6df7-4e6e-a836-fe3f0750d326 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.870395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 174/327] 10.222.0.69 () {60 vars in 1134 bytes} [Wed Jul 20 19:09:54 2022] POST /identity/v3/groups => generated 226 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:09:54.926469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aceb8e7-ba22-43f7-b545-5494e509a5f9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:54.965102 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac67b1e-1a1d-4ca1-ae9e-c6ac2b7fb845 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:54.965762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac67b1e-1a1d-4ca1-ae9e-c6ac2b7fb845 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:54.966048 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac67b1e-1a1d-4ca1-ae9e-c6ac2b7fb845 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:54.966861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac67b1e-1a1d-4ca1-ae9e-c6ac2b7fb845 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:54.967710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac67b1e-1a1d-4ca1-ae9e-c6ac2b7fb845 None admin] RBAC: Adding query filter params (role.id=91f9d5c6716547c18da989bf327f6aca, user.id=b0f4b7e1333c4648ba9ddb08e70c45b4, scope.system=all) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:54.968028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac67b1e-1a1d-4ca1-ae9e-c6ac2b7fb845 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:54.969645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac67b1e-1a1d-4ca1-ae9e-c6ac2b7fb845 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:54.984120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 154/328] 10.222.0.69 () {56 vars in 1308 bytes} [Wed Jul 20 19:09:54 2022] GET /identity/v3/role_assignments?user.id=b0f4b7e1333c4648ba9ddb08e70c45b4&scope.system=all&role.id=91f9d5c6716547c18da989bf327f6aca => generated 510 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:55.001374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fc50e-996e-4420-b368-e89ff936f5be None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.001724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fc50e-996e-4420-b368-e89ff936f5be None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.001978 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9fc50e-996e-4420-b368-e89ff936f5be None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.003627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 175/329] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:55.009484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1fb48-8b4a-4d9c-bbec-04ddd5595c6a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.009829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1fb48-8b4a-4d9c-bbec-04ddd5595c6a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.010078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d1fb48-8b4a-4d9c-bbec-04ddd5595c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.028355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-83d1fb48-8b4a-4d9c-bbec-04ddd5595c6a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:55.030398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-83d1fb48-8b4a-4d9c-bbec-04ddd5595c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:55.106401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 155/330] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:55.128913 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fc50e-996e-4420-b368-e89ff936f5be None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.143681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:55.251174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9fc50e-996e-4420-b368-e89ff936f5be None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.290730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e984d88-229e-428b-8677-670103a3a710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.291472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e984d88-229e-428b-8677-670103a3a710 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.291787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e984d88-229e-428b-8677-670103a3a710 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.292066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e984d88-229e-428b-8677-670103a3a710 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.307146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e984d88-229e-428b-8677-670103a3a710 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-0e984d88-229e-428b-8677-670103a3a710 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:55.316300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:55.317220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:55.318067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 176/331] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:55.323399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1fb48-8b4a-4d9c-bbec-04ddd5595c6a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.346916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:55.440356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d1fb48-8b4a-4d9c-bbec-04ddd5595c6a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.479183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8881797-4a4e-45b9-a3c3-660239d1177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.479863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8881797-4a4e-45b9-a3c3-660239d1177a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.480150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8881797-4a4e-45b9-a3c3-660239d1177a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.480429 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8881797-4a4e-45b9-a3c3-660239d1177a None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.481180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8881797-4a4e-45b9-a3c3-660239d1177a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:55.481489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8881797-4a4e-45b9-a3c3-660239d1177a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:55.483969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8881797-4a4e-45b9-a3c3-660239d1177a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:55.493261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 156/332] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:55.499233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e984d88-229e-428b-8677-670103a3a710 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.583885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:55.620360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e984d88-229e-428b-8677-670103a3a710 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.660130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b4e691-f552-4ace-8327-314be30d29d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.661014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4e691-f552-4ace-8327-314be30d29d5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.661294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4e691-f552-4ace-8327-314be30d29d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.661613 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4e691-f552-4ace-8327-314be30d29d5 None admin] PATH_INFO: `/v3/users/83a6d5a5d95d4c6d969e10a3cd14f78d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.662827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4e691-f552-4ace-8327-314be30d29d5 None admin] RBAC: Authorizing `identity:get_user(user_id=83a6d5a5d95d4c6d969e10a3cd14f78d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:55.664295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46b4e691-f552-4ace-8327-314be30d29d5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:55.665978 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 177/333] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:09:55 2022] GET /identity/v3/users/83a6d5a5d95d4c6d969e10a3cd14f78d => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:55.672437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8881797-4a4e-45b9-a3c3-660239d1177a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:55.773987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:55.801248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8881797-4a4e-45b9-a3c3-660239d1177a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:55.837325 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5382be1e-fe9a-4213-96a0-9e7e5e93dd78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:55.838146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382be1e-fe9a-4213-96a0-9e7e5e93dd78 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:55.838431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382be1e-fe9a-4213-96a0-9e7e5e93dd78 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:55.838709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5382be1e-fe9a-4213-96a0-9e7e5e93dd78 None admin] PATH_INFO: `/v3/system/users/83a6d5a5d95d4c6d969e10a3cd14f78d/roles/997417d78f914e27958b852313dcae0e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:55.840217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382be1e-fe9a-4213-96a0-9e7e5e93dd78 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=83a6d5a5d95d4c6d969e10a3cd14f78d, role_id=997417d78f914e27958b852313dcae0e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:55.842170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5382be1e-fe9a-4213-96a0-9e7e5e93dd78 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:55.852595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 157/334] 10.222.0.69 () {58 vars in 1334 bytes} [Wed Jul 20 19:09:55 2022] PUT /identity/v3/system/users/83a6d5a5d95d4c6d969e10a3cd14f78d/roles/997417d78f914e27958b852313dcae0e => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:09:57.238488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd8138-ad05-44da-8f0c-95e068190e91 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:57.239531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd8138-ad05-44da-8f0c-95e068190e91 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:57.240013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cd8138-ad05-44da-8f0c-95e068190e91 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:57.241858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 178/335] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:57.248212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27239798-69f0-46de-b2ec-343b1657e0e7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:57.249093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27239798-69f0-46de-b2ec-343b1657e0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:57.249638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27239798-69f0-46de-b2ec-343b1657e0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:57.272944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-27239798-69f0-46de-b2ec-343b1657e0e7 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:57.274601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-27239798-69f0-46de-b2ec-343b1657e0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:57.364672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 158/336] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:09:57.390533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd8138-ad05-44da-8f0c-95e068190e91 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:57.511238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cd8138-ad05-44da-8f0c-95e068190e91 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:57.556314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca50f2-0383-4a14-92e1-92c5b4b5a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:57.557122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca50f2-0383-4a14-92e1-92c5b4b5a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:57.557402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca50f2-0383-4a14-92e1-92c5b4b5a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:57.557703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adca50f2-0383-4a14-92e1-92c5b4b5a1b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:57.569378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:57.573587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adca50f2-0383-4a14-92e1-92c5b4b5a1b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-adca50f2-0383-4a14-92e1-92c5b4b5a1b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:57.584902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:57.586068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:57.587063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 179/337] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:57.593441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27239798-69f0-46de-b2ec-343b1657e0e7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:57.689876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:57.724955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27239798-69f0-46de-b2ec-343b1657e0e7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:57.769130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee472ad7-1406-4193-bbf6-d64252f7d235 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:57.769805 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee472ad7-1406-4193-bbf6-d64252f7d235 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:57.770087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee472ad7-1406-4193-bbf6-d64252f7d235 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:57.770407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee472ad7-1406-4193-bbf6-d64252f7d235 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:57.771152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee472ad7-1406-4193-bbf6-d64252f7d235 None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:57.771456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee472ad7-1406-4193-bbf6-d64252f7d235 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:57.772839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee472ad7-1406-4193-bbf6-d64252f7d235 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:57.783849 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 159/338] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:57.789288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca50f2-0383-4a14-92e1-92c5b4b5a1b1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:57.909457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adca50f2-0383-4a14-92e1-92c5b4b5a1b1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:57.953842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518a07-01f5-406f-b954-1f050a7a17b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:57.954495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518a07-01f5-406f-b954-1f050a7a17b0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:57.954775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518a07-01f5-406f-b954-1f050a7a17b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:57.955048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16518a07-01f5-406f-b954-1f050a7a17b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.013960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff98ede-fa29-479e-b559-391748be084f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.014454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff98ede-fa29-479e-b559-391748be084f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.014853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff98ede-fa29-479e-b559-391748be084f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.016349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 160/339] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:09:58.022074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b681435-b8f5-4fbe-9aaa-53396d30a223 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.022331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b681435-b8f5-4fbe-9aaa-53396d30a223 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.022601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b681435-b8f5-4fbe-9aaa-53396d30a223 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.059706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16518a07-01f5-406f-b954-1f050a7a17b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:58.062250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16518a07-01f5-406f-b954-1f050a7a17b0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:58.062573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0b681435-b8f5-4fbe-9aaa-53396d30a223 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:58.063777 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0b681435-b8f5-4fbe-9aaa-53396d30a223 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:58.064228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:58.222576 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 180/340] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:58.228512 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518a07-01f5-406f-b954-1f050a7a17b0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.251092 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 161/341] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:58.253388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7e45-9992-44bf-8aec-95e17c030580 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.257276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7e45-9992-44bf-8aec-95e17c030580 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.257877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0a7e45-9992-44bf-8aec-95e17c030580 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.259630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 162/342] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:58 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:58.275298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806f493-98e1-4726-b420-a2e9c0d254fc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.275980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806f493-98e1-4726-b420-a2e9c0d254fc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.276460 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806f493-98e1-4726-b420-a2e9c0d254fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.319772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2806f493-98e1-4726-b420-a2e9c0d254fc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:58.328016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2806f493-98e1-4726-b420-a2e9c0d254fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:58.441191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:58.451960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16518a07-01f5-406f-b954-1f050a7a17b0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:58.524385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:58.545360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a5bed-3998-4792-a1f2-ba41d277295f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:58.546188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a5bed-3998-4792-a1f2-ba41d277295f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.546471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a5bed-3998-4792-a1f2-ba41d277295f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.547695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0a5bed-3998-4792-a1f2-ba41d277295f None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.547799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 163/343] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:58.548544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806f493-98e1-4726-b420-a2e9c0d254fc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.549732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a5bed-3998-4792-a1f2-ba41d277295f None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:58.551199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0a5bed-3998-4792-a1f2-ba41d277295f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:58.553667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 181/344] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:09:58 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:58.563916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a5bed-3998-4792-a1f2-ba41d277295f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.776245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0a5bed-3998-4792-a1f2-ba41d277295f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:58.828267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3672-e9ea-49e1-bafe-1160f733c405 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:58.829028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3672-e9ea-49e1-bafe-1160f733c405 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.829312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3672-e9ea-49e1-bafe-1160f733c405 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.829620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6d3672-e9ea-49e1-bafe-1160f733c405 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.836513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d3672-e9ea-49e1-bafe-1160f733c405 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:58.837797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806f493-98e1-4726-b420-a2e9c0d254fc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:58.838063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6d3672-e9ea-49e1-bafe-1160f733c405 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:58.840081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 182/345] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:09:58 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:09:58.840868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3672-e9ea-49e1-bafe-1160f733c405 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.882174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:58.904232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec869b21-4786-4c14-8864-d869203541a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:58.905162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec869b21-4786-4c14-8864-d869203541a3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:58.905453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec869b21-4786-4c14-8864-d869203541a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:58.905969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec869b21-4786-4c14-8864-d869203541a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:58.924041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec869b21-4786-4c14-8864-d869203541a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-ec869b21-4786-4c14-8864-d869203541a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:58.939541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:58.940521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:09:58.941323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 164/346] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:09:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:58.942026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec869b21-4786-4c14-8864-d869203541a3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:58.967833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6d3672-e9ea-49e1-bafe-1160f733c405 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.008724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ed1e2-aa65-4b0d-b612-cdeb9f08b471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.009648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ed1e2-aa65-4b0d-b612-cdeb9f08b471 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.010081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ed1e2-aa65-4b0d-b612-cdeb9f08b471 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.010354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614ed1e2-aa65-4b0d-b612-cdeb9f08b471 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.024091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614ed1e2-aa65-4b0d-b612-cdeb9f08b471 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-614ed1e2-aa65-4b0d-b612-cdeb9f08b471 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:09:59.034571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:09:59.036185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:09:59.037079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 183/347] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:09:58 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:09:59.039303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ed1e2-aa65-4b0d-b612-cdeb9f08b471 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.067070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.141247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec869b21-4786-4c14-8864-d869203541a3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.158961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.172009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614ed1e2-aa65-4b0d-b612-cdeb9f08b471 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.189551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372084d-fcec-4db1-9bd7-98f85fe34460 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.190425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372084d-fcec-4db1-9bd7-98f85fe34460 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.190827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372084d-fcec-4db1-9bd7-98f85fe34460 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.191204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b372084d-fcec-4db1-9bd7-98f85fe34460 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.192580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372084d-fcec-4db1-9bd7-98f85fe34460 None admin] RBAC: Adding query filter params (group.id=6e446b5320c141a7b2ae49c3c1316b4d, role.id=91f9d5c6716547c18da989bf327f6aca, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:59.192999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372084d-fcec-4db1-9bd7-98f85fe34460 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.194631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b372084d-fcec-4db1-9bd7-98f85fe34460 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.203088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 165/348] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Jul 20 19:09:58 2022] GET /identity/v3/role_assignments?group.id=6e446b5320c141a7b2ae49c3c1316b4d&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=91f9d5c6716547c18da989bf327f6aca => generated 260 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.204065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372084d-fcec-4db1-9bd7-98f85fe34460 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.214966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b26ea68-11da-4b21-87ff-35c7a75791c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.215710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ea68-11da-4b21-87ff-35c7a75791c8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.217168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ea68-11da-4b21-87ff-35c7a75791c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.217677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b26ea68-11da-4b21-87ff-35c7a75791c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.218822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ea68-11da-4b21-87ff-35c7a75791c8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:59.219132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ea68-11da-4b21-87ff-35c7a75791c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.221781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b26ea68-11da-4b21-87ff-35c7a75791c8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.235520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 184/349] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.237294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cbaa3-1640-4a61-801a-97385c5553c0 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.237601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cbaa3-1640-4a61-801a-97385c5553c0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.238213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873cbaa3-1640-4a61-801a-97385c5553c0 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.242538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 185/350] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:59 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:59.248476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cbaa3-1640-4a61-801a-97385c5553c0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.366120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b372084d-fcec-4db1-9bd7-98f85fe34460 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.409217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.441650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53b64ff-2572-407a-8a24-56d487df9742 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.442329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b64ff-2572-407a-8a24-56d487df9742 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.442613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b64ff-2572-407a-8a24-56d487df9742 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.442888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b64ff-2572-407a-8a24-56d487df9742 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.443640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b64ff-2572-407a-8a24-56d487df9742 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:09:59.443949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b64ff-2572-407a-8a24-56d487df9742 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.445324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53b64ff-2572-407a-8a24-56d487df9742 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.448633 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873cbaa3-1640-4a61-801a-97385c5553c0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.464067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 166/351] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:09:59.465406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6944e3-2b04-496e-bf0e-3d7f910dbf3f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.465718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6944e3-2b04-496e-bf0e-3d7f910dbf3f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.466018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6944e3-2b04-496e-bf0e-3d7f910dbf3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.498640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5f6944e3-2b04-496e-bf0e-3d7f910dbf3f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:09:59.500021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5f6944e3-2b04-496e-bf0e-3d7f910dbf3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:09:59.514256 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-976cb8a1-d85d-4870-89f9-a74fb52f0495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.514912 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cb8a1-d85d-4870-89f9-a74fb52f0495 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.515344 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cb8a1-d85d-4870-89f9-a74fb52f0495 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.515620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-976cb8a1-d85d-4870-89f9-a74fb52f0495 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.549893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.629274 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976cb8a1-d85d-4870-89f9-a74fb52f0495 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.630792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-976cb8a1-d85d-4870-89f9-a74fb52f0495 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.651021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 167/352] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:09:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:09:59.658514 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6944e3-2b04-496e-bf0e-3d7f910dbf3f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.798451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 186/353] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 550 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:09:59.800141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffb4b2-f634-4b84-809b-285c94a133b4 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.800434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffb4b2-f634-4b84-809b-285c94a133b4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.800713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ffb4b2-f634-4b84-809b-285c94a133b4 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.802158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 187/354] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:09:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:09:59.802947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffb4b2-f634-4b84-809b-285c94a133b4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:09:59.814163 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6944e3-2b04-496e-bf0e-3d7f910dbf3f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.827088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.855058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be07621e-05bc-4776-945b-7f8ca5bf705f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.855719 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07621e-05bc-4776-945b-7f8ca5bf705f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.856000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07621e-05bc-4776-945b-7f8ca5bf705f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.856272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be07621e-05bc-4776-945b-7f8ca5bf705f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:09:59.870093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:09:59.937265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ffb4b2-f634-4b84-809b-285c94a133b4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:09:59.949996 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07621e-05bc-4776-945b-7f8ca5bf705f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:09:59.952226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be07621e-05bc-4776-945b-7f8ca5bf705f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:09:59.985157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a25038f-3ff5-407a-8d87-7f2b3c903c1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:09:59.986067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25038f-3ff5-407a-8d87-7f2b3c903c1f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:09:59.986352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25038f-3ff5-407a-8d87-7f2b3c903c1f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:09:59.988185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a25038f-3ff5-407a-8d87-7f2b3c903c1f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.001335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a25038f-3ff5-407a-8d87-7f2b3c903c1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-3a25038f-3ff5-407a-8d87-7f2b3c903c1f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:00.011141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:00.012038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:00.013444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:00.015045 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 188/355] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:00.016243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563581e-7498-4e42-b56f-4385d5ce9227 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.016243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563581e-7498-4e42-b56f-4385d5ce9227 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.016243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2563581e-7498-4e42-b56f-4385d5ce9227 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.018318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 189/356] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:00.018794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563581e-7498-4e42-b56f-4385d5ce9227 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.104414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 168/357] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:09:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 453 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:00.105655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3cda1-9796-4215-8c51-dfd56794613f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.105946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3cda1-9796-4215-8c51-dfd56794613f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.106223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b3cda1-9796-4215-8c51-dfd56794613f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.124946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-00b3cda1-9796-4215-8c51-dfd56794613f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:00.126239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-00b3cda1-9796-4215-8c51-dfd56794613f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:00.129228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.158544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2563581e-7498-4e42-b56f-4385d5ce9227 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.177742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.208245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04db731-4c77-47ba-8a66-728c84db2ea5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.209579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04db731-4c77-47ba-8a66-728c84db2ea5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.210003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04db731-4c77-47ba-8a66-728c84db2ea5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.210417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04db731-4c77-47ba-8a66-728c84db2ea5 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.211653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04db731-4c77-47ba-8a66-728c84db2ea5 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.213099 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04db731-4c77-47ba-8a66-728c84db2ea5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.214864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 190/358] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:00.215706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04db731-4c77-47ba-8a66-728c84db2ea5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.219579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 169/359] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:00.221981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3184fa0-71b2-41c9-9344-9d0d713687a9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.222693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3184fa0-71b2-41c9-9344-9d0d713687a9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.223626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3184fa0-71b2-41c9-9344-9d0d713687a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.242253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f3184fa0-71b2-41c9-9344-9d0d713687a9 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:00.243717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f3184fa0-71b2-41c9-9344-9d0d713687a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:00.327228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04db731-4c77-47ba-8a66-728c84db2ea5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.329223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 170/360] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:00.331385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3184fa0-71b2-41c9-9344-9d0d713687a9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.368790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daeadb6-7797-4830-86a4-48316f94dd37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.369129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daeadb6-7797-4830-86a4-48316f94dd37 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.369409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daeadb6-7797-4830-86a4-48316f94dd37 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.369756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1daeadb6-7797-4830-86a4-48316f94dd37 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.370504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daeadb6-7797-4830-86a4-48316f94dd37 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:00.370861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daeadb6-7797-4830-86a4-48316f94dd37 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.372749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1daeadb6-7797-4830-86a4-48316f94dd37 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.383111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 191/361] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:00.383789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daeadb6-7797-4830-86a4-48316f94dd37 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.411576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.442510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.452096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3184fa0-71b2-41c9-9344-9d0d713687a9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.505367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0fb46-ca84-4712-b6ad-e481fb1dce4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.506746 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0fb46-ca84-4712-b6ad-e481fb1dce4c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.507236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0fb46-ca84-4712-b6ad-e481fb1dce4c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.508591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a0fb46-ca84-4712-b6ad-e481fb1dce4c None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.509019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0fb46-ca84-4712-b6ad-e481fb1dce4c None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.510535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a0fb46-ca84-4712-b6ad-e481fb1dce4c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.512301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 171/362] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:00.513200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0fb46-ca84-4712-b6ad-e481fb1dce4c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.525999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1daeadb6-7797-4830-86a4-48316f94dd37 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.576533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7547e-dc39-462a-8a45-0691bfb562d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.577408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7547e-dc39-462a-8a45-0691bfb562d7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.577736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7547e-dc39-462a-8a45-0691bfb562d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.578464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c7547e-dc39-462a-8a45-0691bfb562d7 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.579680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7547e-dc39-462a-8a45-0691bfb562d7 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.581131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c7547e-dc39-462a-8a45-0691bfb562d7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.582964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 192/363] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:00.587785 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7547e-dc39-462a-8a45-0691bfb562d7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.642468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a0fb46-ca84-4712-b6ad-e481fb1dce4c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.687918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914936c3-b3c2-4a07-a1fe-6c4930c9a96b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.689045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914936c3-b3c2-4a07-a1fe-6c4930c9a96b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.689548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914936c3-b3c2-4a07-a1fe-6c4930c9a96b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.690006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-914936c3-b3c2-4a07-a1fe-6c4930c9a96b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.698826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.705055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-914936c3-b3c2-4a07-a1fe-6c4930c9a96b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:00.708473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c7547e-dc39-462a-8a45-0691bfb562d7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-914936c3-b3c2-4a07-a1fe-6c4930c9a96b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:00.712846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:00.713780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:00.714630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 172/364] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:00.715287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914936c3-b3c2-4a07-a1fe-6c4930c9a96b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.721189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.754295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79771372-9aae-4fc0-866b-d6bdc1e24ef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.755374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79771372-9aae-4fc0-866b-d6bdc1e24ef1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.755675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79771372-9aae-4fc0-866b-d6bdc1e24ef1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.757092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79771372-9aae-4fc0-866b-d6bdc1e24ef1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.775725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79771372-9aae-4fc0-866b-d6bdc1e24ef1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-79771372-9aae-4fc0-866b-d6bdc1e24ef1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:00.785946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:00.787061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:00.788044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:00.792347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 193/365] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:00.792347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79771372-9aae-4fc0-866b-d6bdc1e24ef1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.860447 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-914936c3-b3c2-4a07-a1fe-6c4930c9a96b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.901976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e638ab1-a7bb-40e8-9f6b-d08cb30a9ce7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.902641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e638ab1-a7bb-40e8-9f6b-d08cb30a9ce7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.902930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e638ab1-a7bb-40e8-9f6b-d08cb30a9ce7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.903209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e638ab1-a7bb-40e8-9f6b-d08cb30a9ce7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.925411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79771372-9aae-4fc0-866b-d6bdc1e24ef1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:00.965310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698912b-3787-49d6-932b-030925042158 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:00.966242 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698912b-3787-49d6-932b-030925042158 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:00.966523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698912b-3787-49d6-932b-030925042158 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:00.966878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6698912b-3787-49d6-932b-030925042158 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:00.968132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698912b-3787-49d6-932b-030925042158 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.969580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6698912b-3787-49d6-932b-030925042158 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:00.971519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 194/366] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:00.972299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698912b-3787-49d6-932b-030925042158 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:00.987290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:00.992074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e638ab1-a7bb-40e8-9f6b-d08cb30a9ce7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:00.996231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e638ab1-a7bb-40e8-9f6b-d08cb30a9ce7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.025042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.110873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6698912b-3787-49d6-932b-030925042158 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.127999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 173/367] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:01.130479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e638ab1-a7bb-40e8-9f6b-d08cb30a9ce7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.156070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3107e-ad91-44df-8f0f-1021ef732fc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.157121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3107e-ad91-44df-8f0f-1021ef732fc9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.157654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3107e-ad91-44df-8f0f-1021ef732fc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.158235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd3107e-ad91-44df-8f0f-1021ef732fc9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.159583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3107e-ad91-44df-8f0f-1021ef732fc9 None admin] RBAC: Adding query filter params (group.id=6e446b5320c141a7b2ae49c3c1316b4d, role.id=91f9d5c6716547c18da989bf327f6aca, scope.project.id=edc462e56b784f1eaefd9e0b56312647) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:01.160123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3107e-ad91-44df-8f0f-1021ef732fc9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.161958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd3107e-ad91-44df-8f0f-1021ef732fc9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.168402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 195/368] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:00 2022] GET /identity/v3/role_assignments?group.id=6e446b5320c141a7b2ae49c3c1316b4d&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=91f9d5c6716547c18da989bf327f6aca => generated 260 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.169080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3107e-ad91-44df-8f0f-1021ef732fc9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.270956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.276606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e638ab1-a7bb-40e8-9f6b-d08cb30a9ce7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.306059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd3107e-ad91-44df-8f0f-1021ef732fc9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.308393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.329491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f63d72-b1d1-47e7-aeec-0a63492184c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.329655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f63d72-b1d1-47e7-aeec-0a63492184c2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.329931 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f63d72-b1d1-47e7-aeec-0a63492184c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.330205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f63d72-b1d1-47e7-aeec-0a63492184c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.330971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f63d72-b1d1-47e7-aeec-0a63492184c2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:01.331279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f63d72-b1d1-47e7-aeec-0a63492184c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.333017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f63d72-b1d1-47e7-aeec-0a63492184c2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.345492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 174/369] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.346580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f63d72-b1d1-47e7-aeec-0a63492184c2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.365717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a58dc-14f5-448e-b60f-dcc2ea59f584 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.366704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a58dc-14f5-448e-b60f-dcc2ea59f584 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.366990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a58dc-14f5-448e-b60f-dcc2ea59f584 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.367262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62a58dc-14f5-448e-b60f-dcc2ea59f584 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.368284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a58dc-14f5-448e-b60f-dcc2ea59f584 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:01.368592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a58dc-14f5-448e-b60f-dcc2ea59f584 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.370202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62a58dc-14f5-448e-b60f-dcc2ea59f584 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.382070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 196/370] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.382988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a58dc-14f5-448e-b60f-dcc2ea59f584 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.466098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f63d72-b1d1-47e7-aeec-0a63492184c2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.504125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1d46a-c270-48ec-9cd0-fc2a2c7edb7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.505166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1d46a-c270-48ec-9cd0-fc2a2c7edb7a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.505690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1d46a-c270-48ec-9cd0-fc2a2c7edb7a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.506179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b1d46a-c270-48ec-9cd0-fc2a2c7edb7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.507875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1d46a-c270-48ec-9cd0-fc2a2c7edb7a None admin] RBAC: Adding query filter params (group.id=6e446b5320c141a7b2ae49c3c1316b4d, role.id=77052d58af4f4b5d9b2d003531462e50, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:01.508422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1d46a-c270-48ec-9cd0-fc2a2c7edb7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.510688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b1d46a-c270-48ec-9cd0-fc2a2c7edb7a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.511440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62a58dc-14f5-448e-b60f-dcc2ea59f584 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.518634 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 175/371] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:01 2022] GET /identity/v3/role_assignments?group.id=6e446b5320c141a7b2ae49c3c1316b4d&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=77052d58af4f4b5d9b2d003531462e50 => generated 260 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.519353 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1d46a-c270-48ec-9cd0-fc2a2c7edb7a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.555576 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70103693-7730-40be-ba2b-72baa816911c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.556593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70103693-7730-40be-ba2b-72baa816911c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.556879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70103693-7730-40be-ba2b-72baa816911c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.557153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70103693-7730-40be-ba2b-72baa816911c None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.558429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70103693-7730-40be-ba2b-72baa816911c None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.559391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.560857 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70103693-7730-40be-ba2b-72baa816911c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.564164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 197/372] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:01 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.565030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70103693-7730-40be-ba2b-72baa816911c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.603901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.658857 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b1d46a-c270-48ec-9cd0-fc2a2c7edb7a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.681555 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70103693-7730-40be-ba2b-72baa816911c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.701076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fa50c7-4355-4bc7-be97-7d4f1b3e53f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.702672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa50c7-4355-4bc7-be97-7d4f1b3e53f3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.702672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa50c7-4355-4bc7-be97-7d4f1b3e53f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.702672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fa50c7-4355-4bc7-be97-7d4f1b3e53f3 None admin] PATH_INFO: `/v3/users/83a6d5a5d95d4c6d969e10a3cd14f78d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.704208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa50c7-4355-4bc7-be97-7d4f1b3e53f3 None admin] RBAC: Authorizing `identity:get_user(user_id=83a6d5a5d95d4c6d969e10a3cd14f78d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.705847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80fa50c7-4355-4bc7-be97-7d4f1b3e53f3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.708103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 176/373] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:10:01 2022] GET /identity/v3/users/83a6d5a5d95d4c6d969e10a3cd14f78d => generated 293 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.709366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a37a5-ee01-47c0-a14e-c8c92377b375 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.713608 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a37a5-ee01-47c0-a14e-c8c92377b375 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.713608 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70a37a5-ee01-47c0-a14e-c8c92377b375 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.713608 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 177/374] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:01.713608 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a37a5-ee01-47c0-a14e-c8c92377b375 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.729361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8c7c6-1067-48fe-9d72-bcda079bfccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.730054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8c7c6-1067-48fe-9d72-bcda079bfccb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.730441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8c7c6-1067-48fe-9d72-bcda079bfccb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.730659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a8c7c6-1067-48fe-9d72-bcda079bfccb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.809604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8c7c6-1067-48fe-9d72-bcda079bfccb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.811174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a8c7c6-1067-48fe-9d72-bcda079bfccb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.843877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.848713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:01.860922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70a37a5-ee01-47c0-a14e-c8c92377b375 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:01.899703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be07dd0-3a47-4430-9f47-b192f28c48a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:01.900844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be07dd0-3a47-4430-9f47-b192f28c48a1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.901281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be07dd0-3a47-4430-9f47-b192f28c48a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.901755 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be07dd0-3a47-4430-9f47-b192f28c48a1 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.903932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be07dd0-3a47-4430-9f47-b192f28c48a1 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:01.905790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be07dd0-3a47-4430-9f47-b192f28c48a1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:01.907755 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 178/375] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:01 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:01.909075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd16661-86d1-4c58-a1f1-05dddeeedaee None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:01.912634 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd16661-86d1-4c58-a1f1-05dddeeedaee None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:01.913292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd16661-86d1-4c58-a1f1-05dddeeedaee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:01.920997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 198/376] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:01.921915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8c7c6-1067-48fe-9d72-bcda079bfccb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:01.934449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-9dd16661-86d1-4c58-a1f1-05dddeeedaee None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:01.935982 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9dd16661-86d1-4c58-a1f1-05dddeeedaee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:02.012852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 179/377] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:02.014191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd16661-86d1-4c58-a1f1-05dddeeedaee None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:02.027448 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a8c7c6-1067-48fe-9d72-bcda079bfccb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:02.087433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:02.094256 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b99f9-177e-48bd-ab10-148631ed6e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:02.094640 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b99f9-177e-48bd-ab10-148631ed6e48 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.094921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b99f9-177e-48bd-ab10-148631ed6e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.095456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b99f9-177e-48bd-ab10-148631ed6e48 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.096626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b99f9-177e-48bd-ab10-148631ed6e48 None admin] RBAC: Adding query filter params (role.id=997417d78f914e27958b852313dcae0e, user.id=83a6d5a5d95d4c6d969e10a3cd14f78d, scope.system=all) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:02.096938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b99f9-177e-48bd-ab10-148631ed6e48 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:02.098693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b99f9-177e-48bd-ab10-148631ed6e48 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:02.111849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 199/378] 10.222.0.69 () {56 vars in 1308 bytes} [Wed Jul 20 19:10:01 2022] GET /identity/v3/role_assignments?user.id=83a6d5a5d95d4c6d969e10a3cd14f78d&scope.system=all&role.id=997417d78f914e27958b852313dcae0e => generated 510 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:02.112687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b99f9-177e-48bd-ab10-148631ed6e48 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:02.130360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:02.151321 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd16661-86d1-4c58-a1f1-05dddeeedaee None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:02.231730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b99f9-177e-48bd-ab10-148631ed6e48 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:02.233216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a33688-31c6-4d6d-8b5a-572a3d44a397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:02.233972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a33688-31c6-4d6d-8b5a-572a3d44a397 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.234256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a33688-31c6-4d6d-8b5a-572a3d44a397 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.234536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a33688-31c6-4d6d-8b5a-572a3d44a397 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.235813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a33688-31c6-4d6d-8b5a-572a3d44a397 None admin] RBAC: Adding query filter params (group.id=6e446b5320c141a7b2ae49c3c1316b4d, role.id=77052d58af4f4b5d9b2d003531462e50, scope.project.id=edc462e56b784f1eaefd9e0b56312647) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:02.235813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a33688-31c6-4d6d-8b5a-572a3d44a397 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:02.237129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a33688-31c6-4d6d-8b5a-572a3d44a397 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:02.246560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 180/379] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:02 2022] GET /identity/v3/role_assignments?group.id=6e446b5320c141a7b2ae49c3c1316b4d&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=77052d58af4f4b5d9b2d003531462e50 => generated 260 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:02.248024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a33688-31c6-4d6d-8b5a-572a3d44a397 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:02.271804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ab229-1f40-4433-9f81-4cdedfec16f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:02.272931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ab229-1f40-4433-9f81-4cdedfec16f5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.273366 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ab229-1f40-4433-9f81-4cdedfec16f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.273803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-497ab229-1f40-4433-9f81-4cdedfec16f5 None admin] PATH_INFO: `/v3/projects/1a14d1da63be4e14a0cb08d341815ffe` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.275265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ab229-1f40-4433-9f81-4cdedfec16f5 None admin] RBAC: Authorizing `identity:get_project(project_id=1a14d1da63be4e14a0cb08d341815ffe)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:02.276961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-497ab229-1f40-4433-9f81-4cdedfec16f5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:02.281127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 200/380] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:02 2022] GET /identity/v3/projects/1a14d1da63be4e14a0cb08d341815ffe => generated 344 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:02.293723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ab229-1f40-4433-9f81-4cdedfec16f5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:02.409213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:02.480428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-497ab229-1f40-4433-9f81-4cdedfec16f5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:02.507381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a33688-31c6-4d6d-8b5a-572a3d44a397 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:02.536415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ae035-a5a5-4f9c-9dbd-1380b0036856 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:02.537807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae035-a5a5-4f9c-9dbd-1380b0036856 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.538256 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae035-a5a5-4f9c-9dbd-1380b0036856 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.538709 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85ae035-a5a5-4f9c-9dbd-1380b0036856 None admin] PATH_INFO: `/v3/groups/5737dc56e1724b4bb8828969c8f9c30a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.545143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ae035-a5a5-4f9c-9dbd-1380b0036856 None admin] RBAC: Authorizing `identity:get_group(group_id=5737dc56e1724b4bb8828969c8f9c30a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:02.547184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85ae035-a5a5-4f9c-9dbd-1380b0036856 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:02.550443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 201/381] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:02 2022] GET /identity/v3/groups/5737dc56e1724b4bb8828969c8f9c30a => generated 229 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:02.556724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ae035-a5a5-4f9c-9dbd-1380b0036856 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:02.579111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6a7fe-7cce-4eeb-ae09-ba300530915e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:02.580022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6a7fe-7cce-4eeb-ae09-ba300530915e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.580373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6a7fe-7cce-4eeb-ae09-ba300530915e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.580714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be6a7fe-7cce-4eeb-ae09-ba300530915e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.612035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be6a7fe-7cce-4eeb-ae09-ba300530915e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1be6a7fe-7cce-4eeb-ae09-ba300530915e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:02.620234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:02.621172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:02.622051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:02.625156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 181/382] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 378 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:02.631138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6a7fe-7cce-4eeb-ae09-ba300530915e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:02.664138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:02.679712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85ae035-a5a5-4f9c-9dbd-1380b0036856 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:02.704484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:02.738897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a426b-1322-4aed-8b59-02cbc35807c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:02.739110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a426b-1322-4aed-8b59-02cbc35807c3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.739386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a426b-1322-4aed-8b59-02cbc35807c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.739664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116a426b-1322-4aed-8b59-02cbc35807c3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.740511 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a426b-1322-4aed-8b59-02cbc35807c3 None admin] RBAC: Adding query filter params (group.id=5737dc56e1724b4bb8828969c8f9c30a, role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=1a14d1da63be4e14a0cb08d341815ffe) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:02.740825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a426b-1322-4aed-8b59-02cbc35807c3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:02.745081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-116a426b-1322-4aed-8b59-02cbc35807c3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:02.748919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 202/383] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:02 2022] GET /identity/v3/role_assignments?group.id=5737dc56e1724b4bb8828969c8f9c30a&scope.project.id=1a14d1da63be4e14a0cb08d341815ffe&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:02.819613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be6a7fe-7cce-4eeb-ae09-ba300530915e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:02.864647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36254be1-102a-43ed-845a-0b9b03b664a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:02.865334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36254be1-102a-43ed-845a-0b9b03b664a6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:02.865639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36254be1-102a-43ed-845a-0b9b03b664a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:02.865919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36254be1-102a-43ed-845a-0b9b03b664a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:02.866693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36254be1-102a-43ed-845a-0b9b03b664a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:02.867086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36254be1-102a-43ed-845a-0b9b03b664a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:02.868532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36254be1-102a-43ed-845a-0b9b03b664a6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:02.881928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 182/384] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:02.887984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a426b-1322-4aed-8b59-02cbc35807c3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:03.015191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116a426b-1322-4aed-8b59-02cbc35807c3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:03.058229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce73d0a-840c-4ece-9ad3-45337af48798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:03.059099 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce73d0a-840c-4ece-9ad3-45337af48798 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:03.059381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce73d0a-840c-4ece-9ad3-45337af48798 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:03.059790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce73d0a-840c-4ece-9ad3-45337af48798 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.061531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce73d0a-840c-4ece-9ad3-45337af48798 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:03.062979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce73d0a-840c-4ece-9ad3-45337af48798 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:03.064947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 203/385] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:02 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:03.070219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36254be1-102a-43ed-845a-0b9b03b664a6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:03.171622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:03.195271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36254be1-102a-43ed-845a-0b9b03b664a6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:03.240046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c8ad03-6fe1-4887-abc9-5df7d5b5c185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:03.240920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8ad03-6fe1-4887-abc9-5df7d5b5c185 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:03.241199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8ad03-6fe1-4887-abc9-5df7d5b5c185 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:03.241475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c8ad03-6fe1-4887-abc9-5df7d5b5c185 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.242694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8ad03-6fe1-4887-abc9-5df7d5b5c185 None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:03.244131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33c8ad03-6fe1-4887-abc9-5df7d5b5c185 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:03.246167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 183/386] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:03 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:03.252553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce73d0a-840c-4ece-9ad3-45337af48798 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:03.330109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:03.371635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce73d0a-840c-4ece-9ad3-45337af48798 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:03.412618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274e763-94b1-4d4c-ae8c-35e24b86fa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:03.413561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274e763-94b1-4d4c-ae8c-35e24b86fa6d None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:03.413813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274e763-94b1-4d4c-ae8c-35e24b86fa6d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:03.414090 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274e763-94b1-4d4c-ae8c-35e24b86fa6d None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/groups/6e446b5320c141a7b2ae49c3c1316b4d/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.416146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274e763-94b1-4d4c-ae8c-35e24b86fa6d None admin] RBAC: Authorizing `identity:create_grant(project_id=a60cff69e3074f9bb2d0f231a1fb2be4, group_id=6e446b5320c141a7b2ae49c3c1316b4d, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:03.417626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274e763-94b1-4d4c-ae8c-35e24b86fa6d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:03.437402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 204/387] 10.222.0.69 () {58 vars in 1442 bytes} [Wed Jul 20 19:10:03 2022] PUT /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/groups/6e446b5320c141a7b2ae49c3c1316b4d/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:03.826455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e034eb-f3b5-41d4-a1d8-00f98adee66f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:03.827001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e034eb-f3b5-41d4-a1d8-00f98adee66f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:03.827001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e034eb-f3b5-41d4-a1d8-00f98adee66f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.828684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 184/388] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:03.835050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41ea6c-64ac-42de-8651-0c97626b0700 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:03.835861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41ea6c-64ac-42de-8651-0c97626b0700 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:03.836412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c41ea6c-64ac-42de-8651-0c97626b0700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:03.855622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4c41ea6c-64ac-42de-8651-0c97626b0700 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:03.857752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4c41ea6c-64ac-42de-8651-0c97626b0700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:03.956170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 205/389] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:03.991370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e034eb-f3b5-41d4-a1d8-00f98adee66f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.108033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e034eb-f3b5-41d4-a1d8-00f98adee66f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.147169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14aeaa19-a1e9-42d2-b77b-d4a271421cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.147974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aeaa19-a1e9-42d2-b77b-d4a271421cfc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.148257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aeaa19-a1e9-42d2-b77b-d4a271421cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.148536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14aeaa19-a1e9-42d2-b77b-d4a271421cfc None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.166183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14aeaa19-a1e9-42d2-b77b-d4a271421cfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-14aeaa19-a1e9-42d2-b77b-d4a271421cfc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:04.174909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:04.175792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:04.176612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 185/390] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:04.185306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41ea6c-64ac-42de-8651-0c97626b0700 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.257264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:04.342145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c41ea6c-64ac-42de-8651-0c97626b0700 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.396668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1a697-c0af-4daf-b331-b5e57c8637ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.397256 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1a697-c0af-4daf-b331-b5e57c8637ec None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.397554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1a697-c0af-4daf-b331-b5e57c8637ec None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.397834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dc1a697-c0af-4daf-b331-b5e57c8637ec None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.398585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1a697-c0af-4daf-b331-b5e57c8637ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:04.398891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1a697-c0af-4daf-b331-b5e57c8637ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:04.400268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dc1a697-c0af-4daf-b331-b5e57c8637ec None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:04.404377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278badb-14d9-4514-9dac-7c1415ad0c08 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.404672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278badb-14d9-4514-9dac-7c1415ad0c08 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.404959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6278badb-14d9-4514-9dac-7c1415ad0c08 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.406726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 186/391] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:04.412207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dfaa5-3407-402c-ab86-4d271b9687eb None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.412496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dfaa5-3407-402c-ab86-4d271b9687eb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.412772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a24dfaa5-3407-402c-ab86-4d271b9687eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.413863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 206/392] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:04.416143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1a697-c0af-4daf-b331-b5e57c8637ec None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.429035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:04.447943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a24dfaa5-3407-402c-ab86-4d271b9687eb None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:04.449147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a24dfaa5-3407-402c-ab86-4d271b9687eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:04.554719 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 187/393] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:04.589844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dfaa5-3407-402c-ab86-4d271b9687eb None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.625055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dc1a697-c0af-4daf-b331-b5e57c8637ec None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.637939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:04.689901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243e4d0-9549-4be6-8060-be94bbe52841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.691122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e4d0-9549-4be6-8060-be94bbe52841 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.691650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e4d0-9549-4be6-8060-be94bbe52841 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.692133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7243e4d0-9549-4be6-8060-be94bbe52841 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.693537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243e4d0-9549-4be6-8060-be94bbe52841 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:04.695279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7243e4d0-9549-4be6-8060-be94bbe52841 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:04.697429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 207/394] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:04.705853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243e4d0-9549-4be6-8060-be94bbe52841 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.771100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a24dfaa5-3407-402c-ab86-4d271b9687eb None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.817472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae09422-4e80-4f10-b8b7-d4d1eeadfaa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.818870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09422-4e80-4f10-b8b7-d4d1eeadfaa9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.820005 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09422-4e80-4f10-b8b7-d4d1eeadfaa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.820656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae09422-4e80-4f10-b8b7-d4d1eeadfaa9 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.824720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:04.839018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae09422-4e80-4f10-b8b7-d4d1eeadfaa9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8ae09422-4e80-4f10-b8b7-d4d1eeadfaa9 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:04.848693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:04.849640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:04.850478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:04.853399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 188/395] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:04.855075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3631e68-b904-4656-b6c6-7e043b6a7661 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.855577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3631e68-b904-4656-b6c6-7e043b6a7661 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.856052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3631e68-b904-4656-b6c6-7e043b6a7661 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.858382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 189/396] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:04.866359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa4f64-a925-4e5b-8cec-4baba6aa89de None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.866359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa4f64-a925-4e5b-8cec-4baba6aa89de None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.866359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30aa4f64-a925-4e5b-8cec-4baba6aa89de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.875316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7243e4d0-9549-4be6-8060-be94bbe52841 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:04.886602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-30aa4f64-a925-4e5b-8cec-4baba6aa89de None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:04.888683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-30aa4f64-a925-4e5b-8cec-4baba6aa89de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:04.917225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73973f-fba1-4245-bbb4-e5815b9b278e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:04.918385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73973f-fba1-4245-bbb4-e5815b9b278e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:04.918673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73973f-fba1-4245-bbb4-e5815b9b278e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:04.919037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d73973f-fba1-4245-bbb4-e5815b9b278e None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:04.920404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73973f-fba1-4245-bbb4-e5815b9b278e None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:04.922477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d73973f-fba1-4245-bbb4-e5815b9b278e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:04.924592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 208/397] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:04.925426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73973f-fba1-4245-bbb4-e5815b9b278e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.977609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 190/398] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:04.977609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa4f64-a925-4e5b-8cec-4baba6aa89de None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:04.999420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:05.085988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30aa4f64-a925-4e5b-8cec-4baba6aa89de None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.089868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d73973f-fba1-4245-bbb4-e5815b9b278e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.113395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:05.126816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26675b-1855-4845-9c58-6aafd6e91e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.127758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26675b-1855-4845-9c58-6aafd6e91e98 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.128046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26675b-1855-4845-9c58-6aafd6e91e98 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.128322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b26675b-1855-4845-9c58-6aafd6e91e98 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647/groups/6e446b5320c141a7b2ae49c3c1316b4d/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.130614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26675b-1855-4845-9c58-6aafd6e91e98 None admin] RBAC: Authorizing `identity:create_grant(project_id=edc462e56b784f1eaefd9e0b56312647, group_id=6e446b5320c141a7b2ae49c3c1316b4d, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.132324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b26675b-1855-4845-9c58-6aafd6e91e98 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.135975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa45826-ed4c-4a5c-b6a4-68e34f1d61cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.137391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa45826-ed4c-4a5c-b6a4-68e34f1d61cd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.138465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa45826-ed4c-4a5c-b6a4-68e34f1d61cd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.138810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa45826-ed4c-4a5c-b6a4-68e34f1d61cd None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.139777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa45826-ed4c-4a5c-b6a4-68e34f1d61cd None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:05.140230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa45826-ed4c-4a5c-b6a4-68e34f1d61cd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.141598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa45826-ed4c-4a5c-b6a4-68e34f1d61cd None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.148643 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 191/399] 10.222.0.69 () {58 vars in 1442 bytes} [Wed Jul 20 19:10:04 2022] PUT /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647/groups/6e446b5320c141a7b2ae49c3c1316b4d/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:05.150071 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26675b-1855-4845-9c58-6aafd6e91e98 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.156875 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 209/400] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:05.162592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa45826-ed4c-4a5c-b6a4-68e34f1d61cd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.277342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b26675b-1855-4845-9c58-6aafd6e91e98 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.329197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7c3b37-77d6-4cc9-940b-20be82159822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.330499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c3b37-77d6-4cc9-940b-20be82159822 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.331013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c3b37-77d6-4cc9-940b-20be82159822 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.331464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7c3b37-77d6-4cc9-940b-20be82159822 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.332018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa45826-ed4c-4a5c-b6a4-68e34f1d61cd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.335222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:05.346186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7c3b37-77d6-4cc9-940b-20be82159822 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1e7c3b37-77d6-4cc9-940b-20be82159822 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:05.355044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:05.355998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:05.357444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 192/401] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:05.358380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1a660-124a-4eb7-8214-1605cc4e9985 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.358380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1a660-124a-4eb7-8214-1605cc4e9985 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.358380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71a1a660-124a-4eb7-8214-1605cc4e9985 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.359830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 193/402] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:05.364475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1a660-124a-4eb7-8214-1605cc4e9985 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.376971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da2573f-b2a0-42db-be83-4196822a2107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.378207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2573f-b2a0-42db-be83-4196822a2107 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.378716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2573f-b2a0-42db-be83-4196822a2107 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.379224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da2573f-b2a0-42db-be83-4196822a2107 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.380645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2573f-b2a0-42db-be83-4196822a2107 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.382372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3da2573f-b2a0-42db-be83-4196822a2107 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.384509 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 210/403] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:05.385964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6da789-6147-4ead-b947-b4c73f184a4b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.386475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6da789-6147-4ead-b947-b4c73f184a4b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.386954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6da789-6147-4ead-b947-b4c73f184a4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.406179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2e6da789-6147-4ead-b947-b4c73f184a4b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:05.407779 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2e6da789-6147-4ead-b947-b4c73f184a4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:05.430797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:05.491331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 211/404] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:05.492633 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6da789-6147-4ead-b947-b4c73f184a4b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.520577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71a1a660-124a-4eb7-8214-1605cc4e9985 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.559281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28dfe9-3c1a-4a4b-bc3c-f727ab922bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.560508 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28dfe9-3c1a-4a4b-bc3c-f727ab922bae None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.560800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28dfe9-3c1a-4a4b-bc3c-f727ab922bae None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.561183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be28dfe9-3c1a-4a4b-bc3c-f727ab922bae None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.562018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28dfe9-3c1a-4a4b-bc3c-f727ab922bae None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:05.562338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28dfe9-3c1a-4a4b-bc3c-f727ab922bae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.563803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be28dfe9-3c1a-4a4b-bc3c-f727ab922bae None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.572987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 194/405] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:05.573901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28dfe9-3c1a-4a4b-bc3c-f727ab922bae None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.633954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6da789-6147-4ead-b947-b4c73f184a4b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.636396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:05.681903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86deda3d-9fe1-4f4c-a85c-06438c906afe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.682767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deda3d-9fe1-4f4c-a85c-06438c906afe None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.683050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deda3d-9fe1-4f4c-a85c-06438c906afe None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.683326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86deda3d-9fe1-4f4c-a85c-06438c906afe None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.685775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86deda3d-9fe1-4f4c-a85c-06438c906afe None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.687202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86deda3d-9fe1-4f4c-a85c-06438c906afe None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.688197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be28dfe9-3c1a-4a4b-bc3c-f727ab922bae None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.688917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 212/406] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:05.689712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86deda3d-9fe1-4f4c-a85c-06438c906afe None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.730512 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:05.735863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dad47e-c72a-42c6-9af9-37e83139ea0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.737449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dad47e-c72a-42c6-9af9-37e83139ea0d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.738015 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dad47e-c72a-42c6-9af9-37e83139ea0d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.738852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dad47e-c72a-42c6-9af9-37e83139ea0d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.753907 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dad47e-c72a-42c6-9af9-37e83139ea0d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-32dad47e-c72a-42c6-9af9-37e83139ea0d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:05.764672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:05.765722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:05.766526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:05.768960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 195/407] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:05.770040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dad47e-c72a-42c6-9af9-37e83139ea0d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.802460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86deda3d-9fe1-4f4c-a85c-06438c906afe None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.840229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10459143-95f9-41a8-9009-e6c6ccaa9929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.841340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10459143-95f9-41a8-9009-e6c6ccaa9929 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.841466 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10459143-95f9-41a8-9009-e6c6ccaa9929 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.841703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10459143-95f9-41a8-9009-e6c6ccaa9929 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.842987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10459143-95f9-41a8-9009-e6c6ccaa9929 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.844543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10459143-95f9-41a8-9009-e6c6ccaa9929 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.846429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 213/408] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:05.847004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10459143-95f9-41a8-9009-e6c6ccaa9929 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.884501 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dad47e-c72a-42c6-9af9-37e83139ea0d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.890614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:05.924932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c20d8-2f86-4fe3-a971-dc3bfd844102 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:05.926242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c20d8-2f86-4fe3-a971-dc3bfd844102 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:05.926753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c20d8-2f86-4fe3-a971-dc3bfd844102 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:05.927227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c20d8-2f86-4fe3-a971-dc3bfd844102 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/groups/6e446b5320c141a7b2ae49c3c1316b4d/roles/77052d58af4f4b5d9b2d003531462e50` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:05.929905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c20d8-2f86-4fe3-a971-dc3bfd844102 None admin] RBAC: Authorizing `identity:create_grant(project_id=a60cff69e3074f9bb2d0f231a1fb2be4, group_id=6e446b5320c141a7b2ae49c3c1316b4d, role_id=77052d58af4f4b5d9b2d003531462e50)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:05.931698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c20d8-2f86-4fe3-a971-dc3bfd844102 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:05.948645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 196/409] 10.222.0.69 () {58 vars in 1442 bytes} [Wed Jul 20 19:10:05 2022] PUT /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4/groups/6e446b5320c141a7b2ae49c3c1316b4d/roles/77052d58af4f4b5d9b2d003531462e50 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:05.950089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c20d8-2f86-4fe3-a971-dc3bfd844102 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:05.958542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10459143-95f9-41a8-9009-e6c6ccaa9929 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:05.985810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.001670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54e0a9-b626-49a1-9d11-eb724eda4aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.002359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54e0a9-b626-49a1-9d11-eb724eda4aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.002697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54e0a9-b626-49a1-9d11-eb724eda4aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.003028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c54e0a9-b626-49a1-9d11-eb724eda4aa6 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.003808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54e0a9-b626-49a1-9d11-eb724eda4aa6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:06.004120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54e0a9-b626-49a1-9d11-eb724eda4aa6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.011591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c54e0a9-b626-49a1-9d11-eb724eda4aa6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:06.016776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 214/410] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:06.026917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54e0a9-b626-49a1-9d11-eb724eda4aa6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.122314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c20d8-2f86-4fe3-a971-dc3bfd844102 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.156086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.160494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c54e0a9-b626-49a1-9d11-eb724eda4aa6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.173699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44265a98-8b79-42a2-b213-5a00a6acd7e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.174914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44265a98-8b79-42a2-b213-5a00a6acd7e2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.175271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44265a98-8b79-42a2-b213-5a00a6acd7e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.175873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44265a98-8b79-42a2-b213-5a00a6acd7e2 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.177674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44265a98-8b79-42a2-b213-5a00a6acd7e2 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.179548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44265a98-8b79-42a2-b213-5a00a6acd7e2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:06.182394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 197/411] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:05 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:06.191739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44265a98-8b79-42a2-b213-5a00a6acd7e2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.209804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a0807-814b-478d-84ab-bab2d1a0450b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.210673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a0807-814b-478d-84ab-bab2d1a0450b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.210961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a0807-814b-478d-84ab-bab2d1a0450b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.211505 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59a0807-814b-478d-84ab-bab2d1a0450b None admin] PATH_INFO: `/v3/groups/5737dc56e1724b4bb8828969c8f9c30a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.213210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a0807-814b-478d-84ab-bab2d1a0450b None admin] RBAC: Authorizing `identity:get_group(group_id=5737dc56e1724b4bb8828969c8f9c30a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.214993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59a0807-814b-478d-84ab-bab2d1a0450b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:06.217933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 215/412] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:06 2022] GET /identity/v3/groups/5737dc56e1724b4bb8828969c8f9c30a => generated 229 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:06.224100 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a0807-814b-478d-84ab-bab2d1a0450b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.328410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.337093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44265a98-8b79-42a2-b213-5a00a6acd7e2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.356092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59a0807-814b-478d-84ab-bab2d1a0450b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.390260 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105c7bf-c208-4dea-89cf-91d99d2da900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.391657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105c7bf-c208-4dea-89cf-91d99d2da900 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.392190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105c7bf-c208-4dea-89cf-91d99d2da900 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.392677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105c7bf-c208-4dea-89cf-91d99d2da900 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647/groups/6e446b5320c141a7b2ae49c3c1316b4d/roles/77052d58af4f4b5d9b2d003531462e50` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.393389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942577b1-ade5-4c8a-8b78-a506e8dcdb2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.394371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942577b1-ade5-4c8a-8b78-a506e8dcdb2f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.394751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942577b1-ade5-4c8a-8b78-a506e8dcdb2f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.395129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942577b1-ade5-4c8a-8b78-a506e8dcdb2f None admin] PATH_INFO: `/v3/projects/1a14d1da63be4e14a0cb08d341815ffe` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.395792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105c7bf-c208-4dea-89cf-91d99d2da900 None admin] RBAC: Authorizing `identity:create_grant(project_id=edc462e56b784f1eaefd9e0b56312647, group_id=6e446b5320c141a7b2ae49c3c1316b4d, role_id=77052d58af4f4b5d9b2d003531462e50)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.396523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942577b1-ade5-4c8a-8b78-a506e8dcdb2f None admin] RBAC: Authorizing `identity:get_project(project_id=1a14d1da63be4e14a0cb08d341815ffe)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.397571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7105c7bf-c208-4dea-89cf-91d99d2da900 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:06.398294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942577b1-ade5-4c8a-8b78-a506e8dcdb2f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:06.403743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 216/413] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:06 2022] GET /identity/v3/projects/1a14d1da63be4e14a0cb08d341815ffe => generated 344 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:06.406928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc25c-6ba7-4d1a-871b-4e92d1c62574 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.407317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc25c-6ba7-4d1a-871b-4e92d1c62574 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.407683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32bc25c-6ba7-4d1a-871b-4e92d1c62574 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.413401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 217/414] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:06 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:06.414356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc25c-6ba7-4d1a-871b-4e92d1c62574 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.414569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 198/415] 10.222.0.69 () {58 vars in 1442 bytes} [Wed Jul 20 19:10:06 2022] PUT /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647/groups/6e446b5320c141a7b2ae49c3c1316b4d/roles/77052d58af4f4b5d9b2d003531462e50 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:10:06.420821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3066052-c739-4088-b414-2caa39edfd97 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.420999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3066052-c739-4088-b414-2caa39edfd97 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.421283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3066052-c739-4088-b414-2caa39edfd97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.456718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f3066052-c739-4088-b414-2caa39edfd97 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:06.459725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f3066052-c739-4088-b414-2caa39edfd97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:06.500062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.591598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 199/416] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:06.617775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3066052-c739-4088-b414-2caa39edfd97 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.647749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32bc25c-6ba7-4d1a-871b-4e92d1c62574 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.687863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168e032-326a-4fc8-a9cf-e672ae2aaec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.688832 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168e032-326a-4fc8-a9cf-e672ae2aaec2 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.689120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168e032-326a-4fc8-a9cf-e672ae2aaec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.689404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d168e032-326a-4fc8-a9cf-e672ae2aaec2 None admin] PATH_INFO: `/v3/projects/1a14d1da63be4e14a0cb08d341815ffe/groups/5737dc56e1724b4bb8828969c8f9c30a/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.692223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168e032-326a-4fc8-a9cf-e672ae2aaec2 None admin] RBAC: Authorizing `identity:create_grant(project_id=1a14d1da63be4e14a0cb08d341815ffe, group_id=5737dc56e1724b4bb8828969c8f9c30a, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.693767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d168e032-326a-4fc8-a9cf-e672ae2aaec2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:06.713363 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 218/417] 10.222.0.69 () {58 vars in 1442 bytes} [Wed Jul 20 19:10:06 2022] PUT /identity/v3/projects/1a14d1da63be4e14a0cb08d341815ffe/groups/5737dc56e1724b4bb8828969c8f9c30a/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:06.724706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.745886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3066052-c739-4088-b414-2caa39edfd97 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.787966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759bc2b-d677-4cb5-9f4c-54f33c9d6619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.789224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759bc2b-d677-4cb5-9f4c-54f33c9d6619 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.789789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759bc2b-d677-4cb5-9f4c-54f33c9d6619 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.790266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e759bc2b-d677-4cb5-9f4c-54f33c9d6619 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.803214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e759bc2b-d677-4cb5-9f4c-54f33c9d6619 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e759bc2b-d677-4cb5-9f4c-54f33c9d6619 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:06.811697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:06.812652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:06.813461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:06.816328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 200/418] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:06.822027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168e032-326a-4fc8-a9cf-e672ae2aaec2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:06.941795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:06.949965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d168e032-326a-4fc8-a9cf-e672ae2aaec2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:06.994396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed588a-dc48-49a7-b96b-782e25961991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:06.995067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed588a-dc48-49a7-b96b-782e25961991 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:06.995354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed588a-dc48-49a7-b96b-782e25961991 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:06.995638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ed588a-dc48-49a7-b96b-782e25961991 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:06.996388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed588a-dc48-49a7-b96b-782e25961991 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:06.996699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed588a-dc48-49a7-b96b-782e25961991 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:06.998190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ed588a-dc48-49a7-b96b-782e25961991 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:07.007386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 219/419] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:07.012595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759bc2b-d677-4cb5-9f4c-54f33c9d6619 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:07.155750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e759bc2b-d677-4cb5-9f4c-54f33c9d6619 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:07.191399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fe1f783-f59c-4286-b1d8-c4138265f1fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:07.192113 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1f783-f59c-4286-b1d8-c4138265f1fd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.192340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1f783-f59c-4286-b1d8-c4138265f1fd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.192617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fe1f783-f59c-4286-b1d8-c4138265f1fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.207693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:07.263886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1f783-f59c-4286-b1d8-c4138265f1fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:07.265384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fe1f783-f59c-4286-b1d8-c4138265f1fd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:07.376420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 201/420] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:07.382606 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed588a-dc48-49a7-b96b-782e25961991 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:07.540400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ed588a-dc48-49a7-b96b-782e25961991 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:07.583392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dc8f3-9fa7-4fe0-8241-65ea60c4fce2 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.583659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dc8f3-9fa7-4fe0-8241-65ea60c4fce2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.583937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-762dc8f3-9fa7-4fe0-8241-65ea60c4fce2 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.588970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c372379-19db-483b-8f90-4822aa712660 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:07.589267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 202/421] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:07.589936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c372379-19db-483b-8f90-4822aa712660 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.590214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c372379-19db-483b-8f90-4822aa712660 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.590487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c372379-19db-483b-8f90-4822aa712660 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.590674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c09e3-3e07-4b72-8fec-5b5b3bdf7cf3 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.591301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c09e3-3e07-4b72-8fec-5b5b3bdf7cf3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.591624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005c09e3-3e07-4b72-8fec-5b5b3bdf7cf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.592096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c372379-19db-483b-8f90-4822aa712660 None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:07.593730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c372379-19db-483b-8f90-4822aa712660 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:07.595752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 220/422] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:07 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:07.603153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c372379-19db-483b-8f90-4822aa712660 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:07.627573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-005c09e3-3e07-4b72-8fec-5b5b3bdf7cf3 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:07.628769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-005c09e3-3e07-4b72-8fec-5b5b3bdf7cf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:07.633242 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:07.702843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:07.749814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 203/423] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:07.771951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c09e3-3e07-4b72-8fec-5b5b3bdf7cf3 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:07.783458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c372379-19db-483b-8f90-4822aa712660 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:07.846497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff375da-accb-404f-b84b-6ed337cf8a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:07.847274 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff375da-accb-404f-b84b-6ed337cf8a88 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:07.847558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff375da-accb-404f-b84b-6ed337cf8a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:07.847835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff375da-accb-404f-b84b-6ed337cf8a88 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:07.849409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff375da-accb-404f-b84b-6ed337cf8a88 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:07.850868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fff375da-accb-404f-b84b-6ed337cf8a88 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:07.859057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 221/424] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:07 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:07.864515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff375da-accb-404f-b84b-6ed337cf8a88 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:07.976683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005c09e3-3e07-4b72-8fec-5b5b3bdf7cf3 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:07.982552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:07.990439 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fff375da-accb-404f-b84b-6ed337cf8a88 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.025904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e4f25-3f2f-4856-b6af-4d439cf237d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.026914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e4f25-3f2f-4856-b6af-4d439cf237d1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.027434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e4f25-3f2f-4856-b6af-4d439cf237d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.027940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154e4f25-3f2f-4856-b6af-4d439cf237d1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.029719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e4f25-3f2f-4856-b6af-4d439cf237d1 None admin] RBAC: Adding query filter params (group.id=6e446b5320c141a7b2ae49c3c1316b4d, role.id=91f9d5c6716547c18da989bf327f6aca, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:08.030248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e4f25-3f2f-4856-b6af-4d439cf237d1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.031892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154e4f25-3f2f-4856-b6af-4d439cf237d1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.038701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 222/425] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:07 2022] GET /identity/v3/role_assignments?group.id=6e446b5320c141a7b2ae49c3c1316b4d&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=91f9d5c6716547c18da989bf327f6aca => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:08.096423 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be22522-cf42-4d2d-8e89-6a7557d40203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.097299 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be22522-cf42-4d2d-8e89-6a7557d40203 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.097534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be22522-cf42-4d2d-8e89-6a7557d40203 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.097821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be22522-cf42-4d2d-8e89-6a7557d40203 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.119434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be22522-cf42-4d2d-8e89-6a7557d40203 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6be22522-cf42-4d2d-8e89-6a7557d40203 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:08.128388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:08.129367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 20 19:10:08.130215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:08.134505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 204/426] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 363 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:08.140685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e4f25-3f2f-4856-b6af-4d439cf237d1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.256007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154e4f25-3f2f-4856-b6af-4d439cf237d1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.292139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a47e3a-7e28-46d8-809a-23a7fc569391 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.293158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a47e3a-7e28-46d8-809a-23a7fc569391 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.293678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a47e3a-7e28-46d8-809a-23a7fc569391 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.294876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a47e3a-7e28-46d8-809a-23a7fc569391 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.296001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a47e3a-7e28-46d8-809a-23a7fc569391 None admin] RBAC: Adding query filter params (name=member) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:08.296531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a47e3a-7e28-46d8-809a-23a7fc569391 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.298142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a47e3a-7e28-46d8-809a-23a7fc569391 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.307579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 223/427] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:08.314161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be22522-cf42-4d2d-8e89-6a7557d40203 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.374037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.431232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be22522-cf42-4d2d-8e89-6a7557d40203 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.468745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5fbed-9502-4cff-9ddd-79c1273e09c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.469826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5fbed-9502-4cff-9ddd-79c1273e09c4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.470319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5fbed-9502-4cff-9ddd-79c1273e09c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.470799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5fbed-9502-4cff-9ddd-79c1273e09c4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.542965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5fbed-9502-4cff-9ddd-79c1273e09c4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.545654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5fbed-9502-4cff-9ddd-79c1273e09c4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.584167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.608127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3d49f-0914-4bf5-82a3-a2739458bffd None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.608464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3d49f-0914-4bf5-82a3-a2739458bffd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.609025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c3d49f-0914-4bf5-82a3-a2739458bffd None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.610837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 224/428] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:08.617358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b0e80-575e-48db-a4fe-a1559e5dbb2c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.617678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b0e80-575e-48db-a4fe-a1559e5dbb2c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.617957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b0e80-575e-48db-a4fe-a1559e5dbb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.638267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-192b0e80-575e-48db-a4fe-a1559e5dbb2c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:08.639477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-192b0e80-575e-48db-a4fe-a1559e5dbb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:08.656718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.661725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 205/429] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:10:08.666336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5fbed-9502-4cff-9ddd-79c1273e09c4 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.716353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 225/430] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:08.743541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b0e80-575e-48db-a4fe-a1559e5dbb2c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.780484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5fbed-9502-4cff-9ddd-79c1273e09c4 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.819034 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc00c3f-4764-4b38-bc87-6b85a3c1c539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.820241 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc00c3f-4764-4b38-bc87-6b85a3c1c539 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.820726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc00c3f-4764-4b38-bc87-6b85a3c1c539 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.821232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc00c3f-4764-4b38-bc87-6b85a3c1c539 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.822830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc00c3f-4764-4b38-bc87-6b85a3c1c539 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:08.824510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc00c3f-4764-4b38-bc87-6b85a3c1c539 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:08.827109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 206/431] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:08.832354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc00c3f-4764-4b38-bc87-6b85a3c1c539 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.835631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.857087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b0e80-575e-48db-a4fe-a1559e5dbb2c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.898063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53b8d1-ed6d-45a7-bdfd-6c4271e556a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.898919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53b8d1-ed6d-45a7-bdfd-6c4271e556a8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:08.899198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53b8d1-ed6d-45a7-bdfd-6c4271e556a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:08.899539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b53b8d1-ed6d-45a7-bdfd-6c4271e556a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:08.913768 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b53b8d1-ed6d-45a7-bdfd-6c4271e556a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2b53b8d1-ed6d-45a7-bdfd-6c4271e556a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:08.923346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:08.924286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:08.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 226/432] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:08.930837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53b8d1-ed6d-45a7-bdfd-6c4271e556a8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:08.950594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:08.953906 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc00c3f-4764-4b38-bc87-6b85a3c1c539 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:08.998280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a2c27-3c8e-4679-a36b-b0d76834491d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:08.999647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a2c27-3c8e-4679-a36b-b0d76834491d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.001015 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a2c27-3c8e-4679-a36b-b0d76834491d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.001895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf0a2c27-3c8e-4679-a36b-b0d76834491d None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.003244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a2c27-3c8e-4679-a36b-b0d76834491d None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.005231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf0a2c27-3c8e-4679-a36b-b0d76834491d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.007679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 207/433] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.021187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a2c27-3c8e-4679-a36b-b0d76834491d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.073196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b53b8d1-ed6d-45a7-bdfd-6c4271e556a8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.108344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.123871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5adfd91-385c-47a5-a534-7c231d3931b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.124981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5adfd91-385c-47a5-a534-7c231d3931b9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.125469 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5adfd91-385c-47a5-a534-7c231d3931b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.125866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5adfd91-385c-47a5-a534-7c231d3931b9 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.126736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5adfd91-385c-47a5-a534-7c231d3931b9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:09.127138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5adfd91-385c-47a5-a534-7c231d3931b9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.128843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5adfd91-385c-47a5-a534-7c231d3931b9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.139335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 227/434] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:09.140950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fd0a0-fbed-48b7-be66-5c16cc690564 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.141330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fd0a0-fbed-48b7-be66-5c16cc690564 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.141722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6fd0a0-fbed-48b7-be66-5c16cc690564 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.143470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 228/435] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:09.144306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fd0a0-fbed-48b7-be66-5c16cc690564 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.145239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf0a2c27-3c8e-4679-a36b-b0d76834491d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.190446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83094e47-ccdc-4840-842e-fa3367ee2ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.191059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83094e47-ccdc-4840-842e-fa3367ee2ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.191346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83094e47-ccdc-4840-842e-fa3367ee2ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.191623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83094e47-ccdc-4840-842e-fa3367ee2ae9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.192456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83094e47-ccdc-4840-842e-fa3367ee2ae9 None admin] RBAC: Adding query filter params (group.id=6e446b5320c141a7b2ae49c3c1316b4d, role.id=91f9d5c6716547c18da989bf327f6aca, scope.project.id=edc462e56b784f1eaefd9e0b56312647) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:09.192777 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83094e47-ccdc-4840-842e-fa3367ee2ae9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.194252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83094e47-ccdc-4840-842e-fa3367ee2ae9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.207172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 208/436] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/role_assignments?group.id=6e446b5320c141a7b2ae49c3c1316b4d&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=91f9d5c6716547c18da989bf327f6aca => generated 611 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.207172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df8156-b3b9-4a68-b000-efa5c5ab02ac None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.207172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df8156-b3b9-4a68-b000-efa5c5ab02ac None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.207172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4df8156-b3b9-4a68-b000-efa5c5ab02ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.226815 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f4df8156-b3b9-4a68-b000-efa5c5ab02ac None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:09.228028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f4df8156-b3b9-4a68-b000-efa5c5ab02ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:09.247685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.285080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6fd0a0-fbed-48b7-be66-5c16cc690564 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.319854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 209/437] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:09.321340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e137a-01fb-4ebd-885f-1050c132e109 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.321668 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e137a-01fb-4ebd-885f-1050c132e109 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.321947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948e137a-01fb-4ebd-885f-1050c132e109 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.323374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 210/438] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:09.331570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6996f96-3be7-43e9-a4a8-69ba2946bbc4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.331888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6996f96-3be7-43e9-a4a8-69ba2946bbc4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.332132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6996f96-3be7-43e9-a4a8-69ba2946bbc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.341423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7de40-aabc-45c3-8f2c-126fcedc33b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.342264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7de40-aabc-45c3-8f2c-126fcedc33b9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.342550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7de40-aabc-45c3-8f2c-126fcedc33b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.342826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d7de40-aabc-45c3-8f2c-126fcedc33b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.358122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a6996f96-3be7-43e9-a4a8-69ba2946bbc4 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:09.359531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a6996f96-3be7-43e9-a4a8-69ba2946bbc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:09.424105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7de40-aabc-45c3-8f2c-126fcedc33b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.425739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d7de40-aabc-45c3-8f2c-126fcedc33b9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.431453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.441117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 211/439] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:09.441933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6996f96-3be7-43e9-a4a8-69ba2946bbc4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.506257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.536927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 229/440] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:09.537787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7de40-aabc-45c3-8f2c-126fcedc33b9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.542712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6996f96-3be7-43e9-a4a8-69ba2946bbc4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.577965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5c1ca-6d12-4096-8d99-d143848070f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.578777 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5c1ca-6d12-4096-8d99-d143848070f1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.579067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5c1ca-6d12-4096-8d99-d143848070f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.579348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a5c1ca-6d12-4096-8d99-d143848070f1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.589329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a5c1ca-6d12-4096-8d99-d143848070f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e8a5c1ca-6d12-4096-8d99-d143848070f1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:09.598064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:09.599374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 212/441] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:09.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5c1ca-6d12-4096-8d99-d143848070f1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.641877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d7de40-aabc-45c3-8f2c-126fcedc33b9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.659922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.675793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19f91c-d2a6-42f3-8493-b5beef069bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.676577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19f91c-d2a6-42f3-8493-b5beef069bf9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.676873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19f91c-d2a6-42f3-8493-b5beef069bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.677129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d19f91c-d2a6-42f3-8493-b5beef069bf9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.687237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d19f91c-d2a6-42f3-8493-b5beef069bf9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2d19f91c-d2a6-42f3-8493-b5beef069bf9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:09.694834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:09.695818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:09.696636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 230/442] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:09.697288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19f91c-d2a6-42f3-8493-b5beef069bf9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.710575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a5c1ca-6d12-4096-8d99-d143848070f1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.728124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.745226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f466-c632-449e-af73-5a09b84f3f27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.746143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f466-c632-449e-af73-5a09b84f3f27 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.746436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f466-c632-449e-af73-5a09b84f3f27 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.746714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f466-c632-449e-af73-5a09b84f3f27 None admin] PATH_INFO: `/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.748162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84f466-c632-449e-af73-5a09b84f3f27 None admin] RBAC: Authorizing `identity:get_project(project_id=a60cff69e3074f9bb2d0f231a1fb2be4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.749675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84f466-c632-449e-af73-5a09b84f3f27 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.751444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 213/443] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/projects/a60cff69e3074f9bb2d0f231a1fb2be4 => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.752136 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f466-c632-449e-af73-5a09b84f3f27 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.801241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d19f91c-d2a6-42f3-8493-b5beef069bf9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.835597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72c515-e10a-47b6-895e-9a6cfe815046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.836262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72c515-e10a-47b6-895e-9a6cfe815046 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.836544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72c515-e10a-47b6-895e-9a6cfe815046 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.836822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc72c515-e10a-47b6-895e-9a6cfe815046 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.837595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72c515-e10a-47b6-895e-9a6cfe815046 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:09.837907 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72c515-e10a-47b6-895e-9a6cfe815046 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.839272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc72c515-e10a-47b6-895e-9a6cfe815046 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.847870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 231/444] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.848935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72c515-e10a-47b6-895e-9a6cfe815046 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.861011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f466-c632-449e-af73-5a09b84f3f27 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.883302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.895727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14656f-7a7e-4a89-a6c4-34e2ff200b72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.896397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14656f-7a7e-4a89-a6c4-34e2ff200b72 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.896679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14656f-7a7e-4a89-a6c4-34e2ff200b72 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.896958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc14656f-7a7e-4a89-a6c4-34e2ff200b72 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.897727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14656f-7a7e-4a89-a6c4-34e2ff200b72 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:09.898035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14656f-7a7e-4a89-a6c4-34e2ff200b72 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.899411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc14656f-7a7e-4a89-a6c4-34e2ff200b72 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.908090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 214/445] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.909165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14656f-7a7e-4a89-a6c4-34e2ff200b72 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:09.951972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:09.959115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc72c515-e10a-47b6-895e-9a6cfe815046 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:09.992571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843417c-715c-4ab9-8c56-257530c14fe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:09.993408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843417c-715c-4ab9-8c56-257530c14fe7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:09.993722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843417c-715c-4ab9-8c56-257530c14fe7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:09.993993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f843417c-715c-4ab9-8c56-257530c14fe7 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:09.995155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843417c-715c-4ab9-8c56-257530c14fe7 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:09.996582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f843417c-715c-4ab9-8c56-257530c14fe7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:09.998799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 232/446] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:09.999551 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843417c-715c-4ab9-8c56-257530c14fe7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.009676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc14656f-7a7e-4a89-a6c4-34e2ff200b72 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.042844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3609f-90f7-4953-b433-7655de5ed5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.043487 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3609f-90f7-4953-b433-7655de5ed5fd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.043799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3609f-90f7-4953-b433-7655de5ed5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.044162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3609f-90f7-4953-b433-7655de5ed5fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.095518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:10.102459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f843417c-715c-4ab9-8c56-257530c14fe7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.112492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3609f-90f7-4953-b433-7655de5ed5fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.114048 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e3609f-90f7-4953-b433-7655de5ed5fd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.135864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65324cd5-19e0-4d8b-a58b-a360f39afd56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.136517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65324cd5-19e0-4d8b-a58b-a360f39afd56 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.136786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65324cd5-19e0-4d8b-a58b-a360f39afd56 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.137069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65324cd5-19e0-4d8b-a58b-a360f39afd56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.168816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:10.208808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65324cd5-19e0-4d8b-a58b-a360f39afd56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.210230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65324cd5-19e0-4d8b-a58b-a360f39afd56 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.213311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 215/447] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:10.214028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3609f-90f7-4953-b433-7655de5ed5fd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.298294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:10.314519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 233/448] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:10.314519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65324cd5-19e0-4d8b-a58b-a360f39afd56 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.315480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3609f-90f7-4953-b433-7655de5ed5fd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.350993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c870f58-9c77-43aa-82c7-ee043b5f60d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.351659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c870f58-9c77-43aa-82c7-ee043b5f60d8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.351942 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c870f58-9c77-43aa-82c7-ee043b5f60d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.352218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c870f58-9c77-43aa-82c7-ee043b5f60d8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.353059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c870f58-9c77-43aa-82c7-ee043b5f60d8 None admin] RBAC: Adding query filter params (group.id=6e446b5320c141a7b2ae49c3c1316b4d, role.id=77052d58af4f4b5d9b2d003531462e50, scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:10.353378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c870f58-9c77-43aa-82c7-ee043b5f60d8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.354861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c870f58-9c77-43aa-82c7-ee043b5f60d8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.361570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 216/449] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:10 2022] GET /identity/v3/role_assignments?group.id=6e446b5320c141a7b2ae49c3c1316b4d&scope.project.id=a60cff69e3074f9bb2d0f231a1fb2be4&role.id=77052d58af4f4b5d9b2d003531462e50 => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:10.362118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c870f58-9c77-43aa-82c7-ee043b5f60d8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.396280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:10.447393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65324cd5-19e0-4d8b-a58b-a360f39afd56 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.480900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c870f58-9c77-43aa-82c7-ee043b5f60d8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.488391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7584c1-ba85-4645-bf2a-c827330a0597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.489273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7584c1-ba85-4645-bf2a-c827330a0597 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.489592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7584c1-ba85-4645-bf2a-c827330a0597 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.489876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7584c1-ba85-4645-bf2a-c827330a0597 None admin] PATH_INFO: `/v3/projects/edc462e56b784f1eaefd9e0b56312647` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.491153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7584c1-ba85-4645-bf2a-c827330a0597 None admin] RBAC: Authorizing `identity:get_project(project_id=edc462e56b784f1eaefd9e0b56312647)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.492632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7584c1-ba85-4645-bf2a-c827330a0597 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.499674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 234/450] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:10 2022] GET /identity/v3/projects/edc462e56b784f1eaefd9e0b56312647 => generated 302 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:10.501838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7584c1-ba85-4645-bf2a-c827330a0597 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.523734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498eaff-5996-4626-93d8-54f49f55dd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.524600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498eaff-5996-4626-93d8-54f49f55dd0a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.524884 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498eaff-5996-4626-93d8-54f49f55dd0a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.525164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f498eaff-5996-4626-93d8-54f49f55dd0a None admin] PATH_INFO: `/v3/projects/1a14d1da63be4e14a0cb08d341815ffe` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.526377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498eaff-5996-4626-93d8-54f49f55dd0a None admin] RBAC: Authorizing `identity:get_project(project_id=1a14d1da63be4e14a0cb08d341815ffe)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.527825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f498eaff-5996-4626-93d8-54f49f55dd0a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.531238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 217/451] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:10:10 2022] GET /identity/v3/projects/1a14d1da63be4e14a0cb08d341815ffe => generated 344 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:10.536640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498eaff-5996-4626-93d8-54f49f55dd0a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.556231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:10.632483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7584c1-ba85-4645-bf2a-c827330a0597 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.662255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f498eaff-5996-4626-93d8-54f49f55dd0a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.664060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:10.671645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921d0fc-a8b5-48b9-b3b3-cc79cd9096e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.672505 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d0fc-a8b5-48b9-b3b3-cc79cd9096e2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.672792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d0fc-a8b5-48b9-b3b3-cc79cd9096e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.673066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921d0fc-a8b5-48b9-b3b3-cc79cd9096e2 None admin] PATH_INFO: `/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.674301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921d0fc-a8b5-48b9-b3b3-cc79cd9096e2 None admin] RBAC: Authorizing `identity:get_group(group_id=6e446b5320c141a7b2ae49c3c1316b4d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.675731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c921d0fc-a8b5-48b9-b3b3-cc79cd9096e2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.677544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 235/452] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:10 2022] GET /identity/v3/groups/6e446b5320c141a7b2ae49c3c1316b4d => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:10.683355 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921d0fc-a8b5-48b9-b3b3-cc79cd9096e2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.703788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a88bfc-9e24-4bb5-8b7f-9329cf786e76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.704655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a88bfc-9e24-4bb5-8b7f-9329cf786e76 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.704940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a88bfc-9e24-4bb5-8b7f-9329cf786e76 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.705217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a88bfc-9e24-4bb5-8b7f-9329cf786e76 None admin] PATH_INFO: `/v3/groups/5737dc56e1724b4bb8828969c8f9c30a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.706455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a88bfc-9e24-4bb5-8b7f-9329cf786e76 None admin] RBAC: Authorizing `identity:get_group(group_id=5737dc56e1724b4bb8828969c8f9c30a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.707914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a88bfc-9e24-4bb5-8b7f-9329cf786e76 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.709591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 218/453] 10.222.0.69 () {56 vars in 1180 bytes} [Wed Jul 20 19:10:10 2022] GET /identity/v3/groups/5737dc56e1724b4bb8828969c8f9c30a => generated 229 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:10.717093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a88bfc-9e24-4bb5-8b7f-9329cf786e76 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:10.799977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921d0fc-a8b5-48b9-b3b3-cc79cd9096e2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.822356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:10.843445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a88bfc-9e24-4bb5-8b7f-9329cf786e76 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:10.846320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861426b-c732-4dc3-82e0-c995c33a778c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.846902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861426b-c732-4dc3-82e0-c995c33a778c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.847190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861426b-c732-4dc3-82e0-c995c33a778c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.847470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861426b-c732-4dc3-82e0-c995c33a778c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.848571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861426b-c732-4dc3-82e0-c995c33a778c None admin] RBAC: Adding query filter params (group.id=6e446b5320c141a7b2ae49c3c1316b4d, role.id=77052d58af4f4b5d9b2d003531462e50, scope.project.id=edc462e56b784f1eaefd9e0b56312647) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:10.848898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861426b-c732-4dc3-82e0-c995c33a778c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.850434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861426b-c732-4dc3-82e0-c995c33a778c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.856928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 236/454] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:10 2022] GET /identity/v3/role_assignments?group.id=6e446b5320c141a7b2ae49c3c1316b4d&scope.project.id=edc462e56b784f1eaefd9e0b56312647&role.id=77052d58af4f4b5d9b2d003531462e50 => generated 611 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:10.890463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da4ce112-482a-4865-9345-41c2faf42364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:10.891145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ce112-482a-4865-9345-41c2faf42364 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:10.891431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ce112-482a-4865-9345-41c2faf42364 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:10.891710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da4ce112-482a-4865-9345-41c2faf42364 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:10.892627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ce112-482a-4865-9345-41c2faf42364 None admin] RBAC: Adding query filter params (group.id=5737dc56e1724b4bb8828969c8f9c30a, role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=1a14d1da63be4e14a0cb08d341815ffe) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:10.892989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ce112-482a-4865-9345-41c2faf42364 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:10.894597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da4ce112-482a-4865-9345-41c2faf42364 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:10.901446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 219/455] 10.222.0.69 () {56 vars in 1376 bytes} [Wed Jul 20 19:10:10 2022] GET /identity/v3/role_assignments?group.id=5737dc56e1724b4bb8828969c8f9c30a&scope.project.id=1a14d1da63be4e14a0cb08d341815ffe&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 611 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:13.552938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567bf84-67f9-44b3-8483-960ec611e403 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.553451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567bf84-67f9-44b3-8483-960ec611e403 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.553580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c567bf84-67f9-44b3-8483-960ec611e403 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.555139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 237/456] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:13.561145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb34367-0200-412f-9cf7-50cf77d7af30 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.562000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb34367-0200-412f-9cf7-50cf77d7af30 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.562459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb34367-0200-412f-9cf7-50cf77d7af30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.586581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ddb34367-0200-412f-9cf7-50cf77d7af30 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:13.588055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ddb34367-0200-412f-9cf7-50cf77d7af30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:13.652780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5baea-7ed0-4a0d-ad52-85a70a88bb52 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.653545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5baea-7ed0-4a0d-ad52-85a70a88bb52 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.654039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff5baea-7ed0-4a0d-ad52-85a70a88bb52 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.655911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 238/457] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:13.661063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd1ba-ea37-40d3-8f71-a4f1dc1f32dd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.661431 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd1ba-ea37-40d3-8f71-a4f1dc1f32dd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.661743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cffd1ba-ea37-40d3-8f71-a4f1dc1f32dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.689090 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4cffd1ba-ea37-40d3-8f71-a4f1dc1f32dd None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:13.690885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4cffd1ba-ea37-40d3-8f71-a4f1dc1f32dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:13.694386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 220/458] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:13.759999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb34367-0200-412f-9cf7-50cf77d7af30 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:13.769891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:13.778650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 239/459] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:13.803537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffd1ba-ea37-40d3-8f71-a4f1dc1f32dd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:13.918432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cffd1ba-ea37-40d3-8f71-a4f1dc1f32dd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:13.927902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:13.970168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6b2b31f-4831-42e9-95c4-16ece7092eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:13.970984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2b31f-4831-42e9-95c4-16ece7092eba None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:13.971266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2b31f-4831-42e9-95c4-16ece7092eba None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:13.971541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6b2b31f-4831-42e9-95c4-16ece7092eba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:13.990203 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb34367-0200-412f-9cf7-50cf77d7af30 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:13.997717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2b31f-4831-42e9-95c4-16ece7092eba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:13.999955 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6b2b31f-4831-42e9-95c4-16ece7092eba None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-e6b2b31f-4831-42e9-95c4-16ece7092eba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:14.020017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:14.020946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:14.021831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:14.021831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:14.021831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:14.021831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:14.023940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 240/460] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:14.029734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca050d-c842-4003-9919-254d8902a94e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.030443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca050d-c842-4003-9919-254d8902a94e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.030934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09ca050d-c842-4003-9919-254d8902a94e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.032958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 241/461] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:14 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:14.036162 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca050d-c842-4003-9919-254d8902a94e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:14.044784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671f34e6-f6c4-4eb7-9cb3-7703761444da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:14.045372 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f34e6-f6c4-4eb7-9cb3-7703761444da None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.045778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f34e6-f6c4-4eb7-9cb3-7703761444da None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.046464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671f34e6-f6c4-4eb7-9cb3-7703761444da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.066588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f34e6-f6c4-4eb7-9cb3-7703761444da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:14.068454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671f34e6-f6c4-4eb7-9cb3-7703761444da None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-671f34e6-f6c4-4eb7-9cb3-7703761444da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:14.099027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:14.100165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:14.101216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:14.101216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:14.101216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:14.101216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:14.103501 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 221/462] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 344 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:14.104884 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539cb36-5541-4c44-9e4c-e6434b257a08 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.105295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539cb36-5541-4c44-9e4c-e6434b257a08 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.105722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d539cb36-5541-4c44-9e4c-e6434b257a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.113844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:14.128197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d539cb36-5541-4c44-9e4c-e6434b257a08 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:14.130079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d539cb36-5541-4c44-9e4c-e6434b257a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:14.180885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09ca050d-c842-4003-9919-254d8902a94e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:14.214687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 222/463] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:14.215927 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539cb36-5541-4c44-9e4c-e6434b257a08 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:14.230986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af99d27-9267-4c9d-9c8f-a06e89851bbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:14.232260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af99d27-9267-4c9d-9c8f-a06e89851bbf None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.232657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af99d27-9267-4c9d-9c8f-a06e89851bbf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.233075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af99d27-9267-4c9d-9c8f-a06e89851bbf None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.236185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af99d27-9267-4c9d-9c8f-a06e89851bbf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:14.236610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af99d27-9267-4c9d-9c8f-a06e89851bbf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:14.242029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af99d27-9267-4c9d-9c8f-a06e89851bbf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:14.258367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 242/464] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:14.259655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af99d27-9267-4c9d-9c8f-a06e89851bbf None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:14.299725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:14.343710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d539cb36-5541-4c44-9e4c-e6434b257a08 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:14.375102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af99d27-9267-4c9d-9c8f-a06e89851bbf None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:14.378262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:14.382461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f29f05-8696-40c4-aad0-903156a60812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:14.383447 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f29f05-8696-40c4-aad0-903156a60812 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.383939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f29f05-8696-40c4-aad0-903156a60812 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.384402 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f29f05-8696-40c4-aad0-903156a60812 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.385924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f29f05-8696-40c4-aad0-903156a60812 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:14.386256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f29f05-8696-40c4-aad0-903156a60812 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:14.388428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f29f05-8696-40c4-aad0-903156a60812 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:14.400148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 223/465] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:14.400871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f29f05-8696-40c4-aad0-903156a60812 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:14.429389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fbbc8a-9609-47cc-be1b-c5635f4fde62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:14.430389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbbc8a-9609-47cc-be1b-c5635f4fde62 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.430670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbbc8a-9609-47cc-be1b-c5635f4fde62 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.431008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fbbc8a-9609-47cc-be1b-c5635f4fde62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.442548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbbc8a-9609-47cc-be1b-c5635f4fde62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:14.444005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fbbc8a-9609-47cc-be1b-c5635f4fde62 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-62fbbc8a-9609-47cc-be1b-c5635f4fde62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:14.465212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:14.466359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:14.467420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:14.467420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:14.467420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:14.467420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:14.467420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 243/466] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:14.468158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66b20d-46a1-4c33-bbcd-9e1d6056d031 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.468451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66b20d-46a1-4c33-bbcd-9e1d6056d031 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.468721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a66b20d-46a1-4c33-bbcd-9e1d6056d031 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.474475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 244/467] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:14.474475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66b20d-46a1-4c33-bbcd-9e1d6056d031 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:14.521737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f29f05-8696-40c4-aad0-903156a60812 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:14.543602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:14.560764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4da5d0-f55e-44f2-83c4-4581e08a2db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:14.561446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4da5d0-f55e-44f2-83c4-4581e08a2db0 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.562031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4da5d0-f55e-44f2-83c4-4581e08a2db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.562031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4da5d0-f55e-44f2-83c4-4581e08a2db0 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.562704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4da5d0-f55e-44f2-83c4-4581e08a2db0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:14.564158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4da5d0-f55e-44f2-83c4-4581e08a2db0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:14.586549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 224/468] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:14 2022] POST /identity/v3/users => generated 248 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:14.587445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4da5d0-f55e-44f2-83c4-4581e08a2db0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:14.608028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a66b20d-46a1-4c33-bbcd-9e1d6056d031 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:14.653986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-493d2e64-78f9-4963-ba5b-324d48e696b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:14.654694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d2e64-78f9-4963-ba5b-324d48e696b7 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.654961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d2e64-78f9-4963-ba5b-324d48e696b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.655222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-493d2e64-78f9-4963-ba5b-324d48e696b7 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.656037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d2e64-78f9-4963-ba5b-324d48e696b7 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:14.657465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-493d2e64-78f9-4963-ba5b-324d48e696b7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:14.685244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 245/469] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:14 2022] POST /identity/v3/users => generated 246 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:14.687791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f0742-a045-4faf-a60b-2ab2e2f937a5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.688301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f0742-a045-4faf-a60b-2ab2e2f937a5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.688778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26f0742-a045-4faf-a60b-2ab2e2f937a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.713471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b26f0742-a045-4faf-a60b-2ab2e2f937a5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:14.716468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b26f0742-a045-4faf-a60b-2ab2e2f937a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:14.729355 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:14.735910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4da5d0-f55e-44f2-83c4-4581e08a2db0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:14.808043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422336b-5273-40c8-b9f9-75888d257181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:14.808728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422336b-5273-40c8-b9f9-75888d257181 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:14.809012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422336b-5273-40c8-b9f9-75888d257181 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:14.810301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2422336b-5273-40c8-b9f9-75888d257181 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:14.810764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422336b-5273-40c8-b9f9-75888d257181 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:14.811076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422336b-5273-40c8-b9f9-75888d257181 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:14.812486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2422336b-5273-40c8-b9f9-75888d257181 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:14.830440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 225/470] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:14.833279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 246/471] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:14.835681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422336b-5273-40c8-b9f9-75888d257181 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:14.870669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f0742-a045-4faf-a60b-2ab2e2f937a5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:14.892057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:14.976747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2422336b-5273-40c8-b9f9-75888d257181 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.013394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26f0742-a045-4faf-a60b-2ab2e2f937a5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.020417 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ecc006-3049-4521-b97c-185f87dc593c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.021095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecc006-3049-4521-b97c-185f87dc593c None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.021376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecc006-3049-4521-b97c-185f87dc593c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.021948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ecc006-3049-4521-b97c-185f87dc593c None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.022375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ecc006-3049-4521-b97c-185f87dc593c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:15.024075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ecc006-3049-4521-b97c-185f87dc593c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:15.053640 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381a97a-d40f-4a31-b118-8a659d5149fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.054409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381a97a-d40f-4a31-b118-8a659d5149fb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.054683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381a97a-d40f-4a31-b118-8a659d5149fb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.056315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7381a97a-d40f-4a31-b118-8a659d5149fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.056500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 226/472] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:14 2022] POST /identity/v3/users => generated 248 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:15.057311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39538a40-c2a3-4542-8d9d-03abc6645a47 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.057961 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39538a40-c2a3-4542-8d9d-03abc6645a47 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.058427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39538a40-c2a3-4542-8d9d-03abc6645a47 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.061743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 227/473] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:15.070810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f7040-83a9-4ba0-b626-3d50f79c7d94 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.072832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f7040-83a9-4ba0-b626-3d50f79c7d94 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.073314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f6f7040-83a9-4ba0-b626-3d50f79c7d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.077339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381a97a-d40f-4a31-b118-8a659d5149fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:15.081723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7381a97a-d40f-4a31-b118-8a659d5149fb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:15.088894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.130557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4f6f7040-83a9-4ba0-b626-3d50f79c7d94 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:15.132438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4f6f7040-83a9-4ba0-b626-3d50f79c7d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-7381a97a-d40f-4a31-b118-8a659d5149fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:15.144619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:15.146269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:15.147920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:15.147920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:15.147920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:15.147920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:15.161666 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 247/474] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:15.171732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381a97a-d40f-4a31-b118-8a659d5149fb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.306898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 228/475] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:15.332109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f7040-83a9-4ba0-b626-3d50f79c7d94 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.364550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.374078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7381a97a-d40f-4a31-b118-8a659d5149fb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.420462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1f7ec-9077-4a0a-a6f7-ec74f13bad67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.421125 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1f7ec-9077-4a0a-a6f7-ec74f13bad67 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.421408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1f7ec-9077-4a0a-a6f7-ec74f13bad67 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.421705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d1f7ec-9077-4a0a-a6f7-ec74f13bad67 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.422431 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1f7ec-9077-4a0a-a6f7-ec74f13bad67 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:15.422735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1f7ec-9077-4a0a-a6f7-ec74f13bad67 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:15.424110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d1f7ec-9077-4a0a-a6f7-ec74f13bad67 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:15.441933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 248/476] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:15.447552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1f7ec-9077-4a0a-a6f7-ec74f13bad67 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.454677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f6f7040-83a9-4ba0-b626-3d50f79c7d94 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.503331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3958e0-5830-4064-9725-90833d08198c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.503992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3958e0-5830-4064-9725-90833d08198c None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.504340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3958e0-5830-4064-9725-90833d08198c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.504691 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3958e0-5830-4064-9725-90833d08198c None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.505902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3958e0-5830-4064-9725-90833d08198c None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:15.507646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3958e0-5830-4064-9725-90833d08198c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:15.516000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.sql.core [None req-cd3958e0-5830-4064-9725-90833d08198c None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 19:10:15.516000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 19:10:15.516000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [parameters: {'id': '90c1630b8b764d64bafa9062cbc96285', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 19:10:15.516000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80211) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 19:10:15.520883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-cd3958e0-5830-4064-9725-90833d08198c None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result.read() Jul 20 19:10:15.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return ref.to_dict() Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:10:15.541100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application yield resource Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise self.value Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application session.commit() Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 19:10:15.542383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.session.flush() Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application compat.raise_( Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 19:10:15.543914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:10:15.545093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result.read() Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:10:15.551315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [parameters: {'id': '90c1630b8b764d64bafa9062cbc96285', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:15.552889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 20 19:10:15.557266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:15.559891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 229/477] 10.222.0.69 () {60 vars in 1131 bytes} [Wed Jul 20 19:10:15 2022] POST /identity/v3/roles => generated 144 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:15.559891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3958e0-5830-4064-9725-90833d08198c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.595773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d1f7ec-9077-4a0a-a6f7-ec74f13bad67 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.647049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314fcf-1f80-4393-9016-5253f4554b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.647798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314fcf-1f80-4393-9016-5253f4554b36 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.647999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314fcf-1f80-4393-9016-5253f4554b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.648275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10314fcf-1f80-4393-9016-5253f4554b36 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.649404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10314fcf-1f80-4393-9016-5253f4554b36 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:15.651111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10314fcf-1f80-4393-9016-5253f4554b36 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:15.679409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 249/478] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:15 2022] POST /identity/v3/users => generated 249 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:15.679961 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.707234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3958e0-5830-4064-9725-90833d08198c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.752239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0455cc25-c4a8-44a4-b449-b4c76532c4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.753365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455cc25-c4a8-44a4-b449-b4c76532c4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.753844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455cc25-c4a8-44a4-b449-b4c76532c4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.754312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0455cc25-c4a8-44a4-b449-b4c76532c4a2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:15.774200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0455cc25-c4a8-44a4-b449-b4c76532c4a2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-0455cc25-c4a8-44a4-b449-b4c76532c4a2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:15.782774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:15.783866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 20 19:10:15.784878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:15.787291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 230/479] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:15.794119 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314fcf-1f80-4393-9016-5253f4554b36 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:15.952532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10314fcf-1f80-4393-9016-5253f4554b36 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:15.967244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:15.998694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbb495e-e55f-4666-b2d9-5d7c2bf9ebf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:15.999366 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb495e-e55f-4666-b2d9-5d7c2bf9ebf7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:15.999649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb495e-e55f-4666-b2d9-5d7c2bf9ebf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:15.999920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbb495e-e55f-4666-b2d9-5d7c2bf9ebf7 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:16.000678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb495e-e55f-4666-b2d9-5d7c2bf9ebf7 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:16.000985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb495e-e55f-4666-b2d9-5d7c2bf9ebf7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:16.002384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbb495e-e55f-4666-b2d9-5d7c2bf9ebf7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:16.011344 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 250/480] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:16.999157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9aa84-c0bd-4dab-a236-270dde3d3754 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:16.999668 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9aa84-c0bd-4dab-a236-270dde3d3754 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:16.999668 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db9aa84-c0bd-4dab-a236-270dde3d3754 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.001324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 231/481] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:17.006621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40f4d3-921d-41d2-9157-96fac2481546 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.007018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40f4d3-921d-41d2-9157-96fac2481546 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.007199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e40f4d3-921d-41d2-9157-96fac2481546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.028151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8e40f4d3-921d-41d2-9157-96fac2481546 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:17.029362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8e40f4d3-921d-41d2-9157-96fac2481546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:17.120552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 251/482] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:17.160761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9aa84-c0bd-4dab-a236-270dde3d3754 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:17.309934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db9aa84-c0bd-4dab-a236-270dde3d3754 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:17.361211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdddee-e63a-44da-a5aa-ddaf1609ccad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:17.362055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdddee-e63a-44da-a5aa-ddaf1609ccad None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.362343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdddee-e63a-44da-a5aa-ddaf1609ccad None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.362627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03fdddee-e63a-44da-a5aa-ddaf1609ccad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.376245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03fdddee-e63a-44da-a5aa-ddaf1609ccad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-03fdddee-e63a-44da-a5aa-ddaf1609ccad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:17.384115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:17.384999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:17.386576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 232/483] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:17.389395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f6537-e28d-44d9-b198-bdcd23de60da None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.390297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f6537-e28d-44d9-b198-bdcd23de60da None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.390297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8f6537-e28d-44d9-b198-bdcd23de60da None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.391675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 252/484] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:17.392622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdddee-e63a-44da-a5aa-ddaf1609ccad None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:17.398209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bc5d6-bf13-4999-b3c0-655ca23b13c3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.398493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bc5d6-bf13-4999-b3c0-655ca23b13c3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.398769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8bc5d6-bf13-4999-b3c0-655ca23b13c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.412053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:17.425250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0a8bc5d6-bf13-4999-b3c0-655ca23b13c3 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:17.426468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0a8bc5d6-bf13-4999-b3c0-655ca23b13c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:17.535472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 253/485] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:17.578971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bc5d6-bf13-4999-b3c0-655ca23b13c3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:17.591526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03fdddee-e63a-44da-a5aa-ddaf1609ccad None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:17.654118 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:17.654518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada57a3-6209-4e4a-80fb-a82bfb4c80d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:17.656411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada57a3-6209-4e4a-80fb-a82bfb4c80d8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.656896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada57a3-6209-4e4a-80fb-a82bfb4c80d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.657371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ada57a3-6209-4e4a-80fb-a82bfb4c80d8 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.664596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada57a3-6209-4e4a-80fb-a82bfb4c80d8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:17.664957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada57a3-6209-4e4a-80fb-a82bfb4c80d8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:17.666415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ada57a3-6209-4e4a-80fb-a82bfb4c80d8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:17.684772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 233/486] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:17.690455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada57a3-6209-4e4a-80fb-a82bfb4c80d8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:17.717436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8bc5d6-bf13-4999-b3c0-655ca23b13c3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:17.755462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc839ca-ef37-49db-8360-2fbb45c261d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:17.756376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc839ca-ef37-49db-8360-2fbb45c261d4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.756664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc839ca-ef37-49db-8360-2fbb45c261d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.756942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc839ca-ef37-49db-8360-2fbb45c261d4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.768824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc839ca-ef37-49db-8360-2fbb45c261d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-edc839ca-ef37-49db-8360-2fbb45c261d4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:17.777131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:17.779917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:17.781988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 254/487] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:17.789535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc839ca-ef37-49db-8360-2fbb45c261d4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:17.831164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:17.901437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ada57a3-6209-4e4a-80fb-a82bfb4c80d8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:17.922638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:17.934908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc839ca-ef37-49db-8360-2fbb45c261d4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:17.955415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73d1de1f-2a23-4f6e-beab-6d760fb031ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:17.956478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1de1f-2a23-4f6e-beab-6d760fb031ba None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.956759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1de1f-2a23-4f6e-beab-6d760fb031ba None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.957428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73d1de1f-2a23-4f6e-beab-6d760fb031ba None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.978836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1de1f-2a23-4f6e-beab-6d760fb031ba None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:17.979798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b7172dd-c008-4a96-a2f9-b9d4c901b6e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:17.980319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73d1de1f-2a23-4f6e-beab-6d760fb031ba None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:17.980509 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7172dd-c008-4a96-a2f9-b9d4c901b6e2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:17.980750 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7172dd-c008-4a96-a2f9-b9d4c901b6e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:17.981659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b7172dd-c008-4a96-a2f9-b9d4c901b6e2 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:17.982428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7172dd-c008-4a96-a2f9-b9d4c901b6e2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:17.982998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7172dd-c008-4a96-a2f9-b9d4c901b6e2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:17.984642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b7172dd-c008-4a96-a2f9-b9d4c901b6e2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.002942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 255/488] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:18.002942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f0e71-93f1-492d-8f44-34b44aeeb7fb None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.002942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f0e71-93f1-492d-8f44-34b44aeeb7fb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.002942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f0e71-93f1-492d-8f44-34b44aeeb7fb None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.002942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 256/489] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:18.004058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f0e71-93f1-492d-8f44-34b44aeeb7fb None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-73d1de1f-2a23-4f6e-beab-6d760fb031ba None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:18.013659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:18.015052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:18.016103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:18.016103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:18.016103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:18.016103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:18.019088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 234/490] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 329 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:18.020584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed1f0c-a2bd-4449-b610-8e31cdbddeb2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.021018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed1f0c-a2bd-4449-b610-8e31cdbddeb2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.021416 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ed1f0c-a2bd-4449-b610-8e31cdbddeb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.043349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f5ed1f0c-a2bd-4449-b610-8e31cdbddeb2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:18.046374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f5ed1f0c-a2bd-4449-b610-8e31cdbddeb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:18.126443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 235/491] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:18.127396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed1f0c-a2bd-4449-b610-8e31cdbddeb2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.151494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f0e71-93f1-492d-8f44-34b44aeeb7fb None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.180263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:18.201652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22f14c-d882-4682-bda5-523f42662692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.202742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22f14c-d882-4682-bda5-523f42662692 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.203234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22f14c-d882-4682-bda5-523f42662692 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.203676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc22f14c-d882-4682-bda5-523f42662692 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.216210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22f14c-d882-4682-bda5-523f42662692 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.218362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc22f14c-d882-4682-bda5-523f42662692 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.237770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ed1f0c-a2bd-4449-b610-8e31cdbddeb2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-dc22f14c-d882-4682-bda5-523f42662692 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:18.239919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:18.240744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:18.241710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:18.241710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:18.241710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:18.241710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:18.249002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 257/492] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:18.250339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22f14c-d882-4682-bda5-523f42662692 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.260318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:18.278238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d1ee6-2f67-49d9-9373-dcc48489e0cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.279190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d1ee6-2f67-49d9-9373-dcc48489e0cc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.279528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d1ee6-2f67-49d9-9373-dcc48489e0cc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.279885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd2d1ee6-2f67-49d9-9373-dcc48489e0cc None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.280695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d1ee6-2f67-49d9-9373-dcc48489e0cc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:18.281075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d1ee6-2f67-49d9-9373-dcc48489e0cc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.282579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd2d1ee6-2f67-49d9-9373-dcc48489e0cc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.299716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 236/493] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:18.300708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d1ee6-2f67-49d9-9373-dcc48489e0cc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.380594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc22f14c-d882-4682-bda5-523f42662692 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.413780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd2d1ee6-2f67-49d9-9373-dcc48489e0cc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.421876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3083b24-1e68-4d38-9d53-3e8062d0e4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.422755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3083b24-1e68-4d38-9d53-3e8062d0e4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.423037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3083b24-1e68-4d38-9d53-3e8062d0e4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.423428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3083b24-1e68-4d38-9d53-3e8062d0e4d9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.425055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:18.445787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3083b24-1e68-4d38-9d53-3e8062d0e4d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-d3083b24-1e68-4d38-9d53-3e8062d0e4d9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:18.455140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:18.456161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:18.456945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 258/494] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:18.457645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3083b24-1e68-4d38-9d53-3e8062d0e4d9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.472392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694a68e4-d6a3-44b6-8467-b55f6fa64042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.473077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a68e4-d6a3-44b6-8467-b55f6fa64042 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.473700 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a68e4-d6a3-44b6-8467-b55f6fa64042 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.473970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694a68e4-d6a3-44b6-8467-b55f6fa64042 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.474757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a68e4-d6a3-44b6-8467-b55f6fa64042 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:18.475408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a68e4-d6a3-44b6-8467-b55f6fa64042 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.476498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694a68e4-d6a3-44b6-8467-b55f6fa64042 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.494441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 237/495] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:18.494441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7c4ed-6818-4270-82a0-934f911cc20b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.494441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7c4ed-6818-4270-82a0-934f911cc20b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.494441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7c4ed-6818-4270-82a0-934f911cc20b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.496377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 238/496] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:18.497783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7c4ed-6818-4270-82a0-934f911cc20b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.560862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:18.603478 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3083b24-1e68-4d38-9d53-3e8062d0e4d9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.635566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7c4ed-6818-4270-82a0-934f911cc20b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.642744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6a635-4a5b-48b8-b823-e434fcc13468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.643542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6a635-4a5b-48b8-b823-e434fcc13468 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.643831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6a635-4a5b-48b8-b823-e434fcc13468 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.644108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db6a635-4a5b-48b8-b823-e434fcc13468 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.652911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6a635-4a5b-48b8-b823-e434fcc13468 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.655154 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db6a635-4a5b-48b8-b823-e434fcc13468 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-4db6a635-4a5b-48b8-b823-e434fcc13468 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:18.675527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:18.676503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:18.677381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 259/497] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/users/glance => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:18.678339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6a635-4a5b-48b8-b823-e434fcc13468 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.693615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a89872-283c-4cfa-a71a-d8bc58c0c993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.694420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a89872-283c-4cfa-a71a-d8bc58c0c993 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.694704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a89872-283c-4cfa-a71a-d8bc58c0c993 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.695087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a89872-283c-4cfa-a71a-d8bc58c0c993 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.695871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a89872-283c-4cfa-a71a-d8bc58c0c993 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:18.696178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a89872-283c-4cfa-a71a-d8bc58c0c993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.697615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a89872-283c-4cfa-a71a-d8bc58c0c993 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.711107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 239/498] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:18.713192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb205ea-c231-4dba-80b8-d868ad639439 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.713551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb205ea-c231-4dba-80b8-d868ad639439 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.713829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb205ea-c231-4dba-80b8-d868ad639439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.738541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-efb205ea-c231-4dba-80b8-d868ad639439 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:18.740058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-efb205ea-c231-4dba-80b8-d868ad639439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:18.770551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:18.821044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 240/499] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:18.822912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb205ea-c231-4dba-80b8-d868ad639439 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.836307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db6a635-4a5b-48b8-b823-e434fcc13468 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.875574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:18.886015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0b37d-6786-4330-90ce-2d936589f89a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.886765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0b37d-6786-4330-90ce-2d936589f89a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.886985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0b37d-6786-4330-90ce-2d936589f89a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.887528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab0b37d-6786-4330-90ce-2d936589f89a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.895893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0b37d-6786-4330-90ce-2d936589f89a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.897383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab0b37d-6786-4330-90ce-2d936589f89a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-fab0b37d-6786-4330-90ce-2d936589f89a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:18.912861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:18.914605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 260/500] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:18.916026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0b37d-6786-4330-90ce-2d936589f89a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:18.954877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb205ea-c231-4dba-80b8-d868ad639439 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:18.991956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92854-620a-4d95-9bd9-c1f750e6d81e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:18.992668 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92854-620a-4d95-9bd9-c1f750e6d81e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:18.992953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92854-620a-4d95-9bd9-c1f750e6d81e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:18.993234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92854-620a-4d95-9bd9-c1f750e6d81e None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:18.994107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af92854-620a-4d95-9bd9-c1f750e6d81e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:18.994425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af92854-620a-4d95-9bd9-c1f750e6d81e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:18.995885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af92854-620a-4d95-9bd9-c1f750e6d81e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.035771 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 241/501] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:19.036759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92854-620a-4d95-9bd9-c1f750e6d81e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.039416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab0b37d-6786-4330-90ce-2d936589f89a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.067375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.078454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2731a3ba-ee5d-4bd2-93a7-dca1240c4978 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.079245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731a3ba-ee5d-4bd2-93a7-dca1240c4978 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.079533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731a3ba-ee5d-4bd2-93a7-dca1240c4978 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.079811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2731a3ba-ee5d-4bd2-93a7-dca1240c4978 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.092020 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731a3ba-ee5d-4bd2-93a7-dca1240c4978 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.093628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2731a3ba-ee5d-4bd2-93a7-dca1240c4978 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2731a3ba-ee5d-4bd2-93a7-dca1240c4978 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:19.112419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:19.113377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:19.114269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:19.114269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:19.114269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:19.114269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:19.114269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 261/502] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:19.115379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5e24a-907c-4931-91c9-006fed5a707a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.115581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5e24a-907c-4931-91c9-006fed5a707a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.115855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee5e24a-907c-4931-91c9-006fed5a707a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.117270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 262/503] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:19.117853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5e24a-907c-4931-91c9-006fed5a707a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.160848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92854-620a-4d95-9bd9-c1f750e6d81e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.161815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.203433 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc06a7c-2189-489f-9136-d1652a5fd7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.204238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06a7c-2189-489f-9136-d1652a5fd7f1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.204519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06a7c-2189-489f-9136-d1652a5fd7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.204791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc06a7c-2189-489f-9136-d1652a5fd7f1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.216812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc06a7c-2189-489f-9136-d1652a5fd7f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8dc06a7c-2189-489f-9136-d1652a5fd7f1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:19.224772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:19.225901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 242/504] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:19.226917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc06a7c-2189-489f-9136-d1652a5fd7f1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.229127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee5e24a-907c-4931-91c9-006fed5a707a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.265474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2719a-cc61-4566-8094-bfa79caf8c13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.266127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2719a-cc61-4566-8094-bfa79caf8c13 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.266413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2719a-cc61-4566-8094-bfa79caf8c13 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.266689 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa2719a-cc61-4566-8094-bfa79caf8c13 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.267520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2719a-cc61-4566-8094-bfa79caf8c13 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:19.267837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2719a-cc61-4566-8094-bfa79caf8c13 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.269260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa2719a-cc61-4566-8094-bfa79caf8c13 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.308655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 263/505] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:19.309776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2719a-cc61-4566-8094-bfa79caf8c13 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.320395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.339705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc06a7c-2189-489f-9136-d1652a5fd7f1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.375008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d93cb-42cb-42ba-8114-49de99852c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.375657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d93cb-42cb-42ba-8114-49de99852c55 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.375940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d93cb-42cb-42ba-8114-49de99852c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.376220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af5d93cb-42cb-42ba-8114-49de99852c55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.416445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.423543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa2719a-cc61-4566-8094-bfa79caf8c13 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.464631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d93cb-42cb-42ba-8114-49de99852c55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.466791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af5d93cb-42cb-42ba-8114-49de99852c55 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.468224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67039751-9c61-4470-8c19-10b9d8570f6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.468893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67039751-9c61-4470-8c19-10b9d8570f6a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.469197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67039751-9c61-4470-8c19-10b9d8570f6a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.469532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67039751-9c61-4470-8c19-10b9d8570f6a None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.470423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67039751-9c61-4470-8c19-10b9d8570f6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:19.470729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67039751-9c61-4470-8c19-10b9d8570f6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.472097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67039751-9c61-4470-8c19-10b9d8570f6a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.484700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 264/506] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:19.486299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8f449-1def-42ca-ae03-63403d3b310e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.486650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8f449-1def-42ca-ae03-63403d3b310e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.486984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a8f449-1def-42ca-ae03-63403d3b310e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.504761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c4a8f449-1def-42ca-ae03-63403d3b310e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:19.506011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c4a8f449-1def-42ca-ae03-63403d3b310e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:19.552261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.572864 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 243/507] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:19.573741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d93cb-42cb-42ba-8114-49de99852c55 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.586304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 265/508] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:19.586304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8f449-1def-42ca-ae03-63403d3b310e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.673615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.688776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af5d93cb-42cb-42ba-8114-49de99852c55 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.692411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a8f449-1def-42ca-ae03-63403d3b310e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.728138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d97e4f-2a0f-4160-87cf-9cf32e5bde1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.728811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d97e4f-2a0f-4160-87cf-9cf32e5bde1a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.729218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d97e4f-2a0f-4160-87cf-9cf32e5bde1a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.729355 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b0250-5c0a-4a41-ab17-89e8e83ad7fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.729461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4d97e4f-2a0f-4160-87cf-9cf32e5bde1a None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.729923 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b0250-5c0a-4a41-ab17-89e8e83ad7fd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.730653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d97e4f-2a0f-4160-87cf-9cf32e5bde1a None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:19.731188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b0250-5c0a-4a41-ab17-89e8e83ad7fd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.731811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d97e4f-2a0f-4160-87cf-9cf32e5bde1a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.732244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192b0250-5c0a-4a41-ab17-89e8e83ad7fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.732333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4d97e4f-2a0f-4160-87cf-9cf32e5bde1a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.740350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 244/509] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:19.741181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d97e4f-2a0f-4160-87cf-9cf32e5bde1a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.797103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.811206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b0250-5c0a-4a41-ab17-89e8e83ad7fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.812715 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192b0250-5c0a-4a41-ab17-89e8e83ad7fd None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.860995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4d97e4f-2a0f-4160-87cf-9cf32e5bde1a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:19.901240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a75e1-5f52-4c84-b697-5966242b1c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:19.902070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a75e1-5f52-4c84-b697-5966242b1c30 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:19.902383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a75e1-5f52-4c84-b697-5966242b1c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:19.902697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89a75e1-5f52-4c84-b697-5966242b1c30 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:19.906165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:19.911464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a75e1-5f52-4c84-b697-5966242b1c30 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:19.913114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89a75e1-5f52-4c84-b697-5966242b1c30 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:19.922182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 266/510] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:19.923001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b0250-5c0a-4a41-ab17-89e8e83ad7fd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e89a75e1-5f52-4c84-b697-5966242b1c30 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:19.929214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:19.930193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:19.931096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 245/511] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/users/cinder => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:19.931681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a75e1-5f52-4c84-b697-5966242b1c30 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.027377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192b0250-5c0a-4a41-ab17-89e8e83ad7fd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.033055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89a75e1-5f52-4c84-b697-5966242b1c30 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.034605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.062258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eed1aa-1c73-433f-8a43-77c4fc41c982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.063053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed1aa-1c73-433f-8a43-77c4fc41c982 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.063338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed1aa-1c73-433f-8a43-77c4fc41c982 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.063613 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eed1aa-1c73-433f-8a43-77c4fc41c982 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.068259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f102ce-8164-4aac-9c72-d423db4c77ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.069053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f102ce-8164-4aac-9c72-d423db4c77ca None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.069338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f102ce-8164-4aac-9c72-d423db4c77ca None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.069650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f102ce-8164-4aac-9c72-d423db4c77ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.073528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eed1aa-1c73-433f-8a43-77c4fc41c982 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.075702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eed1aa-1c73-433f-8a43-77c4fc41c982 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.081217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f102ce-8164-4aac-9c72-d423db4c77ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.082720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f102ce-8164-4aac-9c72-d423db4c77ca None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-a7eed1aa-1c73-433f-8a43-77c4fc41c982 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:20.093598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:20.094438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:20.095305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:20.095305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:20.095305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:20.095305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:20.095305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 267/512] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:20.095571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eed1aa-1c73-433f-8a43-77c4fc41c982 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-33f102ce-8164-4aac-9c72-d423db4c77ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:20.100087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:20.101845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:20.103732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:20.103732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:20.103732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:20.103732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:20.103732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 246/513] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:20.103732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f102ce-8164-4aac-9c72-d423db4c77ca None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.143841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.204247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f102ce-8164-4aac-9c72-d423db4c77ca None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.206729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eed1aa-1c73-433f-8a43-77c4fc41c982 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.240271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb986d-3428-46b5-bd65-7b688a31cbeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.241056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb986d-3428-46b5-bd65-7b688a31cbeb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.241336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb986d-3428-46b5-bd65-7b688a31cbeb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.241649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdb986d-3428-46b5-bd65-7b688a31cbeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.241817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af798c6f-7d02-42cf-ba60-d374adf2c896 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.242601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af798c6f-7d02-42cf-ba60-d374adf2c896 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.242884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af798c6f-7d02-42cf-ba60-d374adf2c896 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.243165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af798c6f-7d02-42cf-ba60-d374adf2c896 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.253831 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb986d-3428-46b5-bd65-7b688a31cbeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.254279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af798c6f-7d02-42cf-ba60-d374adf2c896 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.255313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdb986d-3428-46b5-bd65-7b688a31cbeb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.255720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af798c6f-7d02-42cf-ba60-d374adf2c896 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-3fdb986d-3428-46b5-bd65-7b688a31cbeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:20.272303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:20.273169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:20.274072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:20.274072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:20.274072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:20.274072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-af798c6f-7d02-42cf-ba60-d374adf2c896 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:20.274283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:20.275055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:20.275909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:20.275909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:20.275909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:20.275909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:20.276137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 247/514] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:20.276204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 268/515] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:20.276204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af798c6f-7d02-42cf-ba60-d374adf2c896 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.277767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb986d-3428-46b5-bd65-7b688a31cbeb None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.287148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.372115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.382288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af798c6f-7d02-42cf-ba60-d374adf2c896 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.384125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdb986d-3428-46b5-bd65-7b688a31cbeb None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.416757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94874e0d-e9b5-4a94-a2e7-d613d0f3047f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.417411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94874e0d-e9b5-4a94-a2e7-d613d0f3047f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.417721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94874e0d-e9b5-4a94-a2e7-d613d0f3047f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.417889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a79ed2-a53f-47a6-b4db-e228d8c4e3e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.418008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94874e0d-e9b5-4a94-a2e7-d613d0f3047f None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.418861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79ed2-a53f-47a6-b4db-e228d8c4e3e5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.418861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79ed2-a53f-47a6-b4db-e228d8c4e3e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.419100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a79ed2-a53f-47a6-b4db-e228d8c4e3e5 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.419189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94874e0d-e9b5-4a94-a2e7-d613d0f3047f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:20.419552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94874e0d-e9b5-4a94-a2e7-d613d0f3047f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.420160 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a79ed2-a53f-47a6-b4db-e228d8c4e3e5 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:20.420472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a79ed2-a53f-47a6-b4db-e228d8c4e3e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.420961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94874e0d-e9b5-4a94-a2e7-d613d0f3047f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.421932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a79ed2-a53f-47a6-b4db-e228d8c4e3e5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.433089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 269/516] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:20.433920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94874e0d-e9b5-4a94-a2e7-d613d0f3047f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.451002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 248/517] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:20.451886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a79ed2-a53f-47a6-b4db-e228d8c4e3e5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.515738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.540312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94874e0d-e9b5-4a94-a2e7-d613d0f3047f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.553683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a79ed2-a53f-47a6-b4db-e228d8c4e3e5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.575243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61812-4a27-46bc-bd27-7e27ef61ba8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.575974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61812-4a27-46bc-bd27-7e27ef61ba8f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.576251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61812-4a27-46bc-bd27-7e27ef61ba8f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.576523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6d61812-4a27-46bc-bd27-7e27ef61ba8f None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.577239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61812-4a27-46bc-bd27-7e27ef61ba8f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:20.577571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61812-4a27-46bc-bd27-7e27ef61ba8f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.578937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6d61812-4a27-46bc-bd27-7e27ef61ba8f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.587721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34490a-c842-44e1-8451-9161f546b53b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.588382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34490a-c842-44e1-8451-9161f546b53b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.588665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34490a-c842-44e1-8451-9161f546b53b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.588941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e34490a-c842-44e1-8451-9161f546b53b None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.589694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34490a-c842-44e1-8451-9161f546b53b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:20.589995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 270/518] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:20.590128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34490a-c842-44e1-8451-9161f546b53b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.590821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61812-4a27-46bc-bd27-7e27ef61ba8f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.591448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e34490a-c842-44e1-8451-9161f546b53b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.603272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 249/519] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:20.603424 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.604172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34490a-c842-44e1-8451-9161f546b53b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.695671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6d61812-4a27-46bc-bd27-7e27ef61ba8f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.708931 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e34490a-c842-44e1-8451-9161f546b53b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.730973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42b9e2-08f0-4593-84d6-389c9aa73e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.731968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42b9e2-08f0-4593-84d6-389c9aa73e33 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.732201 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42b9e2-08f0-4593-84d6-389c9aa73e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.732475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d42b9e2-08f0-4593-84d6-389c9aa73e33 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.733197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42b9e2-08f0-4593-84d6-389c9aa73e33 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:20.733519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42b9e2-08f0-4593-84d6-389c9aa73e33 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.734915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d42b9e2-08f0-4593-84d6-389c9aa73e33 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.735294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.746117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5cb1d-bdd5-449f-971c-9601f9471677 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.747478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5cb1d-bdd5-449f-971c-9601f9471677 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.747763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5cb1d-bdd5-449f-971c-9601f9471677 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.747851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 271/520] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:20.748044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f5cb1d-bdd5-449f-971c-9601f9471677 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.748610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42b9e2-08f0-4593-84d6-389c9aa73e33 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.759195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5cb1d-bdd5-449f-971c-9601f9471677 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.760649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f5cb1d-bdd5-449f-971c-9601f9471677 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-98f5cb1d-bdd5-449f-971c-9601f9471677 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:20.776352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:20.777261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:20.778466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:20.778466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:20.778466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:20.778466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:20.778466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:20.778466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:20.778466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:20.778466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:20.778466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 250/521] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:20.778466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5cb1d-bdd5-449f-971c-9601f9471677 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.820529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:20.853397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d42b9e2-08f0-4593-84d6-389c9aa73e33 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.888152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f5cb1d-bdd5-449f-971c-9601f9471677 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:20.896252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcf1ce-434b-493d-b404-14619b32eb6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.896899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcf1ce-434b-493d-b404-14619b32eb6b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.897188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcf1ce-434b-493d-b404-14619b32eb6b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.897470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fcf1ce-434b-493d-b404-14619b32eb6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.922374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a69b5-a48a-4e7f-b6d4-a74114931924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:20.923036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a69b5-a48a-4e7f-b6d4-a74114931924 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:20.923313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a69b5-a48a-4e7f-b6d4-a74114931924 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:20.923584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a69b5-a48a-4e7f-b6d4-a74114931924 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:20.924271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a69b5-a48a-4e7f-b6d4-a74114931924 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.925738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04a69b5-a48a-4e7f-b6d4-a74114931924 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.946628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 251/522] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:20 2022] POST /identity/v3/users => generated 247 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:20.947444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a69b5-a48a-4e7f-b6d4-a74114931924 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:20.975181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcf1ce-434b-493d-b404-14619b32eb6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:20.977389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fcf1ce-434b-493d-b404-14619b32eb6b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:20.997463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.064761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.074003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04a69b5-a48a-4e7f-b6d4-a74114931924 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.102181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 272/523] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:21.102705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcf1ce-434b-493d-b404-14619b32eb6b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.123629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39edd28-3f61-4a72-b18f-756c201d0eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.124610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39edd28-3f61-4a72-b18f-756c201d0eed None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.124898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39edd28-3f61-4a72-b18f-756c201d0eed None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.125854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39edd28-3f61-4a72-b18f-756c201d0eed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.133704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39edd28-3f61-4a72-b18f-756c201d0eed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.135187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a39edd28-3f61-4a72-b18f-756c201d0eed None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a39edd28-3f61-4a72-b18f-756c201d0eed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:21.150934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:21.151835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:21.152844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:21.152844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:21.152844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:21.152844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:21.152844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:21.152844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:21.152844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:21.152844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:21.152844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 252/524] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:20 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:21.153892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39edd28-3f61-4a72-b18f-756c201d0eed None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.228448 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fcf1ce-434b-493d-b404-14619b32eb6b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.265675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.268271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73403d3d-a12a-4bab-9fd3-b88abb129d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.268976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39edd28-3f61-4a72-b18f-756c201d0eed None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.269093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73403d3d-a12a-4bab-9fd3-b88abb129d49 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.269334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73403d3d-a12a-4bab-9fd3-b88abb129d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.269825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73403d3d-a12a-4bab-9fd3-b88abb129d49 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.277879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73403d3d-a12a-4bab-9fd3-b88abb129d49 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.279320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73403d3d-a12a-4bab-9fd3-b88abb129d49 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-73403d3d-a12a-4bab-9fd3-b88abb129d49 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:21.294228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:21.295667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 273/525] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:21.297195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73403d3d-a12a-4bab-9fd3-b88abb129d49 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.326263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e225cb-3b91-46b0-99ef-7e15dbdb64cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.327393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e225cb-3b91-46b0-99ef-7e15dbdb64cd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.328161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e225cb-3b91-46b0-99ef-7e15dbdb64cd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.328599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e225cb-3b91-46b0-99ef-7e15dbdb64cd None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.329621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e225cb-3b91-46b0-99ef-7e15dbdb64cd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:21.330096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e225cb-3b91-46b0-99ef-7e15dbdb64cd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.332277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e225cb-3b91-46b0-99ef-7e15dbdb64cd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.348728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.363597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 253/526] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:21.363597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e225cb-3b91-46b0-99ef-7e15dbdb64cd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.436115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73403d3d-a12a-4bab-9fd3-b88abb129d49 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.480332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768cfe-e5d8-4b81-99c4-91477d52aabb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.482026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768cfe-e5d8-4b81-99c4-91477d52aabb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.482313 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768cfe-e5d8-4b81-99c4-91477d52aabb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.482608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89768cfe-e5d8-4b81-99c4-91477d52aabb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.493998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89768cfe-e5d8-4b81-99c4-91477d52aabb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.495595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89768cfe-e5d8-4b81-99c4-91477d52aabb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.502446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e225cb-3b91-46b0-99ef-7e15dbdb64cd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-89768cfe-e5d8-4b81-99c4-91477d52aabb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:21.515985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:21.516922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:21.517820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:21.517820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:21.517820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:21.517820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:21.522157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 274/527] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:21.523253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768cfe-e5d8-4b81-99c4-91477d52aabb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.546579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8abfd-88a3-49a0-89f7-ea3ce5d0c177 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.547250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8abfd-88a3-49a0-89f7-ea3ce5d0c177 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.547525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8abfd-88a3-49a0-89f7-ea3ce5d0c177 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.547800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f8abfd-88a3-49a0-89f7-ea3ce5d0c177 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.548583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8abfd-88a3-49a0-89f7-ea3ce5d0c177 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:21.548889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8abfd-88a3-49a0-89f7-ea3ce5d0c177 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.550363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27f8abfd-88a3-49a0-89f7-ea3ce5d0c177 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.571552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 254/528] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:21.572079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8abfd-88a3-49a0-89f7-ea3ce5d0c177 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.595868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.629785 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.655864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89768cfe-e5d8-4b81-99c4-91477d52aabb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.688281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f8abfd-88a3-49a0-89f7-ea3ce5d0c177 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.707836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c19460-eed6-4df6-80e3-e667613a9aa7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.708508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c19460-eed6-4df6-80e3-e667613a9aa7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.708788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c19460-eed6-4df6-80e3-e667613a9aa7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.709060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c19460-eed6-4df6-80e3-e667613a9aa7 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.709922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c19460-eed6-4df6-80e3-e667613a9aa7 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:21.710227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c19460-eed6-4df6-80e3-e667613a9aa7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.711660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74c19460-eed6-4df6-80e3-e667613a9aa7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.725945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7190bf7-d226-4023-80af-7afca7b50c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.726613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7190bf7-d226-4023-80af-7afca7b50c1b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.726918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7190bf7-d226-4023-80af-7afca7b50c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.727193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7190bf7-d226-4023-80af-7afca7b50c1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.728181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7190bf7-d226-4023-80af-7afca7b50c1b None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=3e46ec9c31094d03b6a413e09bc9f699) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:21.728499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7190bf7-d226-4023-80af-7afca7b50c1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.730526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7190bf7-d226-4023-80af-7afca7b50c1b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.745545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 255/529] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/role_assignments?user.id=3e46ec9c31094d03b6a413e09bc9f699&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:21.745763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 275/530] 10.222.0.69 () {56 vars in 1139 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:21.745859 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7190bf7-d226-4023-80af-7afca7b50c1b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.745960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c19460-eed6-4df6-80e3-e667613a9aa7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.874009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.877463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c19460-eed6-4df6-80e3-e667613a9aa7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.899230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7190bf7-d226-4023-80af-7afca7b50c1b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:21.921560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd15a42-eda4-4c91-9e1d-f346fd46cc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.922966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15a42-eda4-4c91-9e1d-f346fd46cc37 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.923397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15a42-eda4-4c91-9e1d-f346fd46cc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.923839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd15a42-eda4-4c91-9e1d-f346fd46cc37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.924821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd15a42-eda4-4c91-9e1d-f346fd46cc37 None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=36955d7d4505457696ab6177adb999a9) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:21.925332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd15a42-eda4-4c91-9e1d-f346fd46cc37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.927874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dd15a42-eda4-4c91-9e1d-f346fd46cc37 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.936010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 276/531] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/role_assignments?user.id=36955d7d4505457696ab6177adb999a9&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:21.936537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd15a42-eda4-4c91-9e1d-f346fd46cc37 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:21.952111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9e1f9-5879-4ab0-b634-01f9a1be3dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:21.953798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9e1f9-5879-4ab0-b634-01f9a1be3dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:21.954696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9e1f9-5879-4ab0-b634-01f9a1be3dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:21.954979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:21.955384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9e1f9-5879-4ab0-b634-01f9a1be3dd9 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:21.956697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9e1f9-5879-4ab0-b634-01f9a1be3dd9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:21.957395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9e1f9-5879-4ab0-b634-01f9a1be3dd9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:21.959384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9e1f9-5879-4ab0-b634-01f9a1be3dd9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:21.979463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 256/532] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:21.986104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9e1f9-5879-4ab0-b634-01f9a1be3dd9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:22.080073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd15a42-eda4-4c91-9e1d-f346fd46cc37 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:22.128084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f919ca-b565-4505-8bec-250a4a6a5696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:22.129278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f919ca-b565-4505-8bec-250a4a6a5696 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.129581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f919ca-b565-4505-8bec-250a4a6a5696 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.130015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f919ca-b565-4505-8bec-250a4a6a5696 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.195337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9e1f9-5879-4ab0-b634-01f9a1be3dd9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:22.250294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f919ca-b565-4505-8bec-250a4a6a5696 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:22.251764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f919ca-b565-4505-8bec-250a4a6a5696 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:22.257534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:22.267525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc24f2-0eda-4b9c-bd48-bbd73df6e75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:22.268342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc24f2-0eda-4b9c-bd48-bbd73df6e75c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.268623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc24f2-0eda-4b9c-bd48-bbd73df6e75c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.268898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2dc24f2-0eda-4b9c-bd48-bbd73df6e75c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.284288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc24f2-0eda-4b9c-bd48-bbd73df6e75c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:22.287429 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2dc24f2-0eda-4b9c-bd48-bbd73df6e75c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-d2dc24f2-0eda-4b9c-bd48-bbd73df6e75c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:22.317154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:22.318084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:22.318934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:22.318934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:22.318934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:22.318934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:22.318934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:22.318934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:22.318934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:22.318934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:22.320952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 257/533] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:22.327455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc24f2-0eda-4b9c-bd48-bbd73df6e75c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:22.337883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:22.448204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 277/534] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 512 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:22.454441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f919ca-b565-4505-8bec-250a4a6a5696 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:22.589536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f919ca-b565-4505-8bec-250a4a6a5696 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:22.589536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:22.632814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2dc24f2-0eda-4b9c-bd48-bbd73df6e75c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:22.643419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53b669-a367-4719-8303-e31530fc1d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:22.644237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53b669-a367-4719-8303-e31530fc1d25 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.644523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53b669-a367-4719-8303-e31530fc1d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.644804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac53b669-a367-4719-8303-e31530fc1d25 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.662300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53b669-a367-4719-8303-e31530fc1d25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:22.663785 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac53b669-a367-4719-8303-e31530fc1d25 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ac53b669-a367-4719-8303-e31530fc1d25 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:22.710239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:22.711120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:22.711955 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:22.711955 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:22.711955 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:22.711955 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:22.716818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 278/535] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:22.718218 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53b669-a367-4719-8303-e31530fc1d25 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:22.758269 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bca61-ca06-40ee-8c2c-95ea005a007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:22.758968 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bca61-ca06-40ee-8c2c-95ea005a007b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.759254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bca61-ca06-40ee-8c2c-95ea005a007b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.759534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b55bca61-ca06-40ee-8c2c-95ea005a007b None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.760336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bca61-ca06-40ee-8c2c-95ea005a007b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:22.760647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bca61-ca06-40ee-8c2c-95ea005a007b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:22.762762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b55bca61-ca06-40ee-8c2c-95ea005a007b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:22.807882 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 258/536] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 482 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:22.814714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bca61-ca06-40ee-8c2c-95ea005a007b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:22.836454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac53b669-a367-4719-8303-e31530fc1d25 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:22.879708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919d12e-7fa2-4893-813b-ca4e992a87be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:22.880382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919d12e-7fa2-4893-813b-ca4e992a87be None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:22.880671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919d12e-7fa2-4893-813b-ca4e992a87be None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:22.880949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f919d12e-7fa2-4893-813b-ca4e992a87be None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:22.881882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919d12e-7fa2-4893-813b-ca4e992a87be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:22.882193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919d12e-7fa2-4893-813b-ca4e992a87be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:22.883964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f919d12e-7fa2-4893-813b-ca4e992a87be None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:22.901943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 279/537] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:22.904085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919d12e-7fa2-4893-813b-ca4e992a87be None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:22.924706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:22.989963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.025171 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f919d12e-7fa2-4893-813b-ca4e992a87be None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.065319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d754338-3f00-45bb-8272-03e0e0e9291b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.066308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d754338-3f00-45bb-8272-03e0e0e9291b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.066601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d754338-3f00-45bb-8272-03e0e0e9291b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.066879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d754338-3f00-45bb-8272-03e0e0e9291b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.075330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d754338-3f00-45bb-8272-03e0e0e9291b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.076952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d754338-3f00-45bb-8272-03e0e0e9291b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.083679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b55bca61-ca06-40ee-8c2c-95ea005a007b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-1d754338-3f00-45bb-8272-03e0e0e9291b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:23.093225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:23.094150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:23.094992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:23.094992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:23.094992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:23.094992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:23.094992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:23.094992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:23.094992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:23.094992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:23.099887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 280/538] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:23.101128 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d754338-3f00-45bb-8272-03e0e0e9291b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.190270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8623bba-a11e-4179-9611-81aa0097d1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.190952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8623bba-a11e-4179-9611-81aa0097d1ed None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.191235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8623bba-a11e-4179-9611-81aa0097d1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.191509 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8623bba-a11e-4179-9611-81aa0097d1ed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.192051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.192346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8623bba-a11e-4179-9611-81aa0097d1ed None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=c94d9030f61a4e1f8b3d75ef8925bdd0) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:23.192659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8623bba-a11e-4179-9611-81aa0097d1ed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.194174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8623bba-a11e-4179-9611-81aa0097d1ed None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.216204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 259/539] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:22 2022] GET /identity/v3/role_assignments?user.id=c94d9030f61a4e1f8b3d75ef8925bdd0&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 259 bytes in 402 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.220854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d754338-3f00-45bb-8272-03e0e0e9291b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.259919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aced1b-591d-4b8e-9006-7bca44e9362c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.260536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aced1b-591d-4b8e-9006-7bca44e9362c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.260763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aced1b-591d-4b8e-9006-7bca44e9362c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.261049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8aced1b-591d-4b8e-9006-7bca44e9362c None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.261881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aced1b-591d-4b8e-9006-7bca44e9362c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:23.262819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aced1b-591d-4b8e-9006-7bca44e9362c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.264295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8aced1b-591d-4b8e-9006-7bca44e9362c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.275162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11d9f5-fb34-4fde-9df2-2500db139e8b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.275474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11d9f5-fb34-4fde-9df2-2500db139e8b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.275726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df11d9f5-fb34-4fde-9df2-2500db139e8b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.277108 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 260/540] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:23.296024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 281/541] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.298533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aced1b-591d-4b8e-9006-7bca44e9362c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.299796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ab60b-5a76-491d-8ccc-e73ffb2e0b0f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.300085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ab60b-5a76-491d-8ccc-e73ffb2e0b0f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.300363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1ab60b-5a76-491d-8ccc-e73ffb2e0b0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.339965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1b1ab60b-5a76-491d-8ccc-e73ffb2e0b0f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:23.343911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1b1ab60b-5a76-491d-8ccc-e73ffb2e0b0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:23.474112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 261/542] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:23 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:23.481737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8aced1b-591d-4b8e-9006-7bca44e9362c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.498639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ab60b-5a76-491d-8ccc-e73ffb2e0b0f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.531144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7425537-ac75-475c-864d-e199fa2c7a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.531721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7425537-ac75-475c-864d-e199fa2c7a16 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.532002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7425537-ac75-475c-864d-e199fa2c7a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.532281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7425537-ac75-475c-864d-e199fa2c7a16 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.533189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7425537-ac75-475c-864d-e199fa2c7a16 None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=3a1a88089d384964bd9774eda4c5d8a9) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:23.533573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7425537-ac75-475c-864d-e199fa2c7a16 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.535460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7425537-ac75-475c-864d-e199fa2c7a16 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.539711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.542791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 282/543] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/role_assignments?user.id=3a1a88089d384964bd9774eda4c5d8a9&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.635761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:23.703540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1ab60b-5a76-491d-8ccc-e73ffb2e0b0f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.762204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4191d56-896a-4cac-b171-2d2bf7de2da0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.763093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4191d56-896a-4cac-b171-2d2bf7de2da0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.763380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4191d56-896a-4cac-b171-2d2bf7de2da0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.763659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4191d56-896a-4cac-b171-2d2bf7de2da0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.782395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4191d56-896a-4cac-b171-2d2bf7de2da0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e4191d56-896a-4cac-b171-2d2bf7de2da0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:23.790100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:23.790969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:23.791762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:23.797001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 262/544] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:23.798728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7425537-ac75-475c-864d-e199fa2c7a16 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:23.915672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7425537-ac75-475c-864d-e199fa2c7a16 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:23.967853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73284c2d-949d-472c-a97b-21599b46c82c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:23.968649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73284c2d-949d-472c-a97b-21599b46c82c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:23.968943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73284c2d-949d-472c-a97b-21599b46c82c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:23.969218 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73284c2d-949d-472c-a97b-21599b46c82c None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:23.969993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73284c2d-949d-472c-a97b-21599b46c82c None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:23.970303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73284c2d-949d-472c-a97b-21599b46c82c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:23.971730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73284c2d-949d-472c-a97b-21599b46c82c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:23.986377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 283/545] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:23.991972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4191d56-896a-4cac-b171-2d2bf7de2da0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:24.131802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4191d56-896a-4cac-b171-2d2bf7de2da0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:24.161122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:24.187662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59a8282b-03f2-46a3-8ff2-7caaeb184c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:24.188983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8282b-03f2-46a3-8ff2-7caaeb184c5b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.189479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8282b-03f2-46a3-8ff2-7caaeb184c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.190002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59a8282b-03f2-46a3-8ff2-7caaeb184c5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.210694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8282b-03f2-46a3-8ff2-7caaeb184c5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:24.212208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59a8282b-03f2-46a3-8ff2-7caaeb184c5b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-59a8282b-03f2-46a3-8ff2-7caaeb184c5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:24.236823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:24.238202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:24.239041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:24.239041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:24.239041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:24.239041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:24.239041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 263/546] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:24.247403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73284c2d-949d-472c-a97b-21599b46c82c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:24.249090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710c0ee-8e82-4a62-8359-a1b1ae1b6b03 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.249383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710c0ee-8e82-4a62-8359-a1b1ae1b6b03 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.249700 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710c0ee-8e82-4a62-8359-a1b1ae1b6b03 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.251089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 264/547] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:24.257176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b014d-0592-4fbd-8382-bdfb4bf4a646 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.257464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b014d-0592-4fbd-8382-bdfb4bf4a646 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.257769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867b014d-0592-4fbd-8382-bdfb4bf4a646 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.286580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-867b014d-0592-4fbd-8382-bdfb4bf4a646 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:24.287791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-867b014d-0592-4fbd-8382-bdfb4bf4a646 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:24.368211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 265/548] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 20 19:10:24.385954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:24.391545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b014d-0592-4fbd-8382-bdfb4bf4a646 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:24.473716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73284c2d-949d-472c-a97b-21599b46c82c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:24.552068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867b014d-0592-4fbd-8382-bdfb4bf4a646 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:24.554535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:24.594998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60e041-669a-415c-91de-e785547ee04a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:24.596330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60e041-669a-415c-91de-e785547ee04a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.596828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60e041-669a-415c-91de-e785547ee04a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.597123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e60e041-669a-415c-91de-e785547ee04a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.602262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54604ea-5a9d-4402-beca-5cba53f40d25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:24.602951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54604ea-5a9d-4402-beca-5cba53f40d25 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.603240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54604ea-5a9d-4402-beca-5cba53f40d25 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.603517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54604ea-5a9d-4402-beca-5cba53f40d25 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.604261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54604ea-5a9d-4402-beca-5cba53f40d25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:24.604574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54604ea-5a9d-4402-beca-5cba53f40d25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:24.606726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54604ea-5a9d-4402-beca-5cba53f40d25 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:24.611825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e60e041-669a-415c-91de-e785547ee04a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-4e60e041-669a-415c-91de-e785547ee04a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:24.619775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:24.621228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:24.623364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:24.628341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 266/549] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:24.628341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60e041-669a-415c-91de-e785547ee04a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:24.632487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 284/550] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:24.634000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8e9bd-56da-4ebc-9714-0c6a3b2bb283 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.634288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8e9bd-56da-4ebc-9714-0c6a3b2bb283 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.634565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a8e9bd-56da-4ebc-9714-0c6a3b2bb283 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.635865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 285/551] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:24.641150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8e9bd-56da-4ebc-9714-0c6a3b2bb283 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:24.763670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e60e041-669a-415c-91de-e785547ee04a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:24.782390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a8e9bd-56da-4ebc-9714-0c6a3b2bb283 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:24.806523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4819a50-7e33-47a5-bbc3-c7a65ea5a7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:24.807244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4819a50-7e33-47a5-bbc3-c7a65ea5a7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.807534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4819a50-7e33-47a5-bbc3-c7a65ea5a7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.807814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4819a50-7e33-47a5-bbc3-c7a65ea5a7e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.808566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4819a50-7e33-47a5-bbc3-c7a65ea5a7e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:24.809490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4819a50-7e33-47a5-bbc3-c7a65ea5a7e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:24.811330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4819a50-7e33-47a5-bbc3-c7a65ea5a7e0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:24.820157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 267/552] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:24.821623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bf626-a46f-4361-8aa4-9c6e5a281942 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.821907 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bf626-a46f-4361-8aa4-9c6e5a281942 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.822188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031bf626-a46f-4361-8aa4-9c6e5a281942 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.828956 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294e4ab-4022-4063-af51-52d3e630e208 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:24.830327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294e4ab-4022-4063-af51-52d3e630e208 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:24.833870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:24.835030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294e4ab-4022-4063-af51-52d3e630e208 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:24.835714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9294e4ab-4022-4063-af51-52d3e630e208 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:24.842244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-031bf626-a46f-4361-8aa4-9c6e5a281942 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:24.843454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-031bf626-a46f-4361-8aa4-9c6e5a281942 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:24.848160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294e4ab-4022-4063-af51-52d3e630e208 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:24.850153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9294e4ab-4022-4063-af51-52d3e630e208 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:24.862741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-9294e4ab-4022-4063-af51-52d3e630e208 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:24.869929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:24.870947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:24.871895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:24.871895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:24.871895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:24.871895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:24.871895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:24.871895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:24.871895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:24.871895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:24.871895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:24.871895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:24.875137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 286/553] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:24 2022] GET /identity/v3/users/swift => generated 83 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:24.876273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294e4ab-4022-4063-af51-52d3e630e208 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:24.950948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 268/554] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:24.951298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031bf626-a46f-4361-8aa4-9c6e5a281942 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.004270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9294e4ab-4022-4063-af51-52d3e630e208 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.052792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173774e-1ab1-44de-a038-c9e235c760b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.053998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173774e-1ab1-44de-a038-c9e235c760b9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.054537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173774e-1ab1-44de-a038-c9e235c760b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.055031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2173774e-1ab1-44de-a038-c9e235c760b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.067806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173774e-1ab1-44de-a038-c9e235c760b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:25.069847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2173774e-1ab1-44de-a038-c9e235c760b9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:25.090191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-031bf626-a46f-4361-8aa4-9c6e5a281942 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2173774e-1ab1-44de-a038-c9e235c760b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:25.090725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:25.091661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:25.092523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:25.092523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:25.092523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:25.092523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:25.094819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 287/555] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:25.096091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173774e-1ab1-44de-a038-c9e235c760b9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.133237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ded2e-b62a-4a6f-8d60-1775d632878d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.134110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ded2e-b62a-4a6f-8d60-1775d632878d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.134267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ded2e-b62a-4a6f-8d60-1775d632878d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.134540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398ded2e-b62a-4a6f-8d60-1775d632878d None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.135367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ded2e-b62a-4a6f-8d60-1775d632878d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:25.135675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ded2e-b62a-4a6f-8d60-1775d632878d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:25.137559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398ded2e-b62a-4a6f-8d60-1775d632878d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:25.155628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:25.158821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 269/556] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Jul 20 19:10:24 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:25.159774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ded2e-b62a-4a6f-8d60-1775d632878d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.173290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:25.271822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398ded2e-b62a-4a6f-8d60-1775d632878d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.283705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2173774e-1ab1-44de-a038-c9e235c760b9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.307834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7d1cc-ebb8-40ba-a63b-bbe773f7d19d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.308516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7d1cc-ebb8-40ba-a63b-bbe773f7d19d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.309333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7d1cc-ebb8-40ba-a63b-bbe773f7d19d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.309848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c7d1cc-ebb8-40ba-a63b-bbe773f7d19d None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.310736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7d1cc-ebb8-40ba-a63b-bbe773f7d19d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:25.311046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7d1cc-ebb8-40ba-a63b-bbe773f7d19d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:25.312565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c7d1cc-ebb8-40ba-a63b-bbe773f7d19d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:25.329118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 270/557] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:25.330761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7d1cc-ebb8-40ba-a63b-bbe773f7d19d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.331788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875948d4-938e-43de-943c-acc80390f165 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.332871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875948d4-938e-43de-943c-acc80390f165 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.333394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875948d4-938e-43de-943c-acc80390f165 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.333883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-875948d4-938e-43de-943c-acc80390f165 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.346351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-875948d4-938e-43de-943c-acc80390f165 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-875948d4-938e-43de-943c-acc80390f165 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:25.355164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:25.357456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:25.358379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:25.361248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 288/558] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:25.362446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875948d4-938e-43de-943c-acc80390f165 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.425693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:25.460738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c7d1cc-ebb8-40ba-a63b-bbe773f7d19d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.477971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:25.489887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-875948d4-938e-43de-943c-acc80390f165 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.502913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36b397-9244-423b-83e0-acf77df4bdf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.503893 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36b397-9244-423b-83e0-acf77df4bdf7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.504360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36b397-9244-423b-83e0-acf77df4bdf7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.504850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e36b397-9244-423b-83e0-acf77df4bdf7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.532358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b145d9-fbb8-41da-9809-8e2cdf13e77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.533639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b145d9-fbb8-41da-9809-8e2cdf13e77e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.534326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b145d9-fbb8-41da-9809-8e2cdf13e77e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.534837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b145d9-fbb8-41da-9809-8e2cdf13e77e None admin] PATH_INFO: `/v3/users/glance` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.544974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b145d9-fbb8-41da-9809-8e2cdf13e77e None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:25.546834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b145d9-fbb8-41da-9809-8e2cdf13e77e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-51b145d9-fbb8-41da-9809-8e2cdf13e77e None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:25.568161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:25.569248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:25.570677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:25.570677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:25.570677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:25.570677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:25.570677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:25.570677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:25.570677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:25.570677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:25.570677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:10:25.570677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:25.577649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 289/559] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:25 2022] GET /identity/v3/users/glance => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:25.577649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b145d9-fbb8-41da-9809-8e2cdf13e77e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.596620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36b397-9244-423b-83e0-acf77df4bdf7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:25.599583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e36b397-9244-423b-83e0-acf77df4bdf7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:25.698385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:25.720741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b145d9-fbb8-41da-9809-8e2cdf13e77e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.743204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 271/560] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:25.744806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36b397-9244-423b-83e0-acf77df4bdf7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.774803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf2aa0-89ba-413a-b899-b0465f413a00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.775481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf2aa0-89ba-413a-b899-b0465f413a00 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.775817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf2aa0-89ba-413a-b899-b0465f413a00 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.776104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74cf2aa0-89ba-413a-b899-b0465f413a00 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.777471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf2aa0-89ba-413a-b899-b0465f413a00 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:25.778781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf2aa0-89ba-413a-b899-b0465f413a00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:25.780590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74cf2aa0-89ba-413a-b899-b0465f413a00 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:25.793898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 290/561] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:25.795673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf2aa0-89ba-413a-b899-b0465f413a00 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.817809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:25.878024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e36b397-9244-423b-83e0-acf77df4bdf7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.928705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74cf2aa0-89ba-413a-b899-b0465f413a00 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:25.931577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda7cac-8c12-4966-b0c5-80244d8bd237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.932262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda7cac-8c12-4966-b0c5-80244d8bd237 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.932546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda7cac-8c12-4966-b0c5-80244d8bd237 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.932822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dda7cac-8c12-4966-b0c5-80244d8bd237 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.934231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda7cac-8c12-4966-b0c5-80244d8bd237 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:25.934552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda7cac-8c12-4966-b0c5-80244d8bd237 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:25.936438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dda7cac-8c12-4966-b0c5-80244d8bd237 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:25.959278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 272/562] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Jul 20 19:10:25 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:25.960227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda7cac-8c12-4966-b0c5-80244d8bd237 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:25.967999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8452d5-fd58-4c04-8ad2-d1d57a65e8cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:25.968867 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8452d5-fd58-4c04-8ad2-d1d57a65e8cf None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:25.969149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8452d5-fd58-4c04-8ad2-d1d57a65e8cf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:25.969436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8452d5-fd58-4c04-8ad2-d1d57a65e8cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:25.985251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8452d5-fd58-4c04-8ad2-d1d57a65e8cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:25.986844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c8452d5-fd58-4c04-8ad2-d1d57a65e8cf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:25.996990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2c8452d5-fd58-4c04-8ad2-d1d57a65e8cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:26.008756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:26.009731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:26.010910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:26.010910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:26.010910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:26.010910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:26.014953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 291/563] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:26.016248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cf291-b7a5-4682-8019-eefb5938ffeb None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.016540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cf291-b7a5-4682-8019-eefb5938ffeb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.016826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59cf291-b7a5-4682-8019-eefb5938ffeb None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.018601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 292/564] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:26.019399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cf291-b7a5-4682-8019-eefb5938ffeb None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.083675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dda7cac-8c12-4966-b0c5-80244d8bd237 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.102911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.121947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63552399-2b22-4be8-a1ba-e4cf2ce2df98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.122741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63552399-2b22-4be8-a1ba-e4cf2ce2df98 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.123019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63552399-2b22-4be8-a1ba-e4cf2ce2df98 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.123294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63552399-2b22-4be8-a1ba-e4cf2ce2df98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.133924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63552399-2b22-4be8-a1ba-e4cf2ce2df98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.135394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63552399-2b22-4be8-a1ba-e4cf2ce2df98 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.142075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59cf291-b7a5-4682-8019-eefb5938ffeb None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-63552399-2b22-4be8-a1ba-e4cf2ce2df98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:26.155381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:26.156515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:26.157670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:26.157670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:26.157670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:26.157670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:26.157670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 273/565] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:26.157670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63552399-2b22-4be8-a1ba-e4cf2ce2df98 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.184561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dba8f2c-2dbc-4ff6-a59f-bf5d87cc4d50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.185358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8f2c-2dbc-4ff6-a59f-bf5d87cc4d50 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.185683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8f2c-2dbc-4ff6-a59f-bf5d87cc4d50 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.185962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dba8f2c-2dbc-4ff6-a59f-bf5d87cc4d50 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.196422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba8f2c-2dbc-4ff6-a59f-bf5d87cc4d50 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.197914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dba8f2c-2dbc-4ff6-a59f-bf5d87cc4d50 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-9dba8f2c-2dbc-4ff6-a59f-bf5d87cc4d50 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:26.221338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:26.222351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:26.223226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:26.223226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:26.223226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:26.223226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:26.223226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 293/566] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:26.224282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67b859-d8ec-4107-995b-2582bae21db3 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.224571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67b859-d8ec-4107-995b-2582bae21db3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.224849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c67b859-d8ec-4107-995b-2582bae21db3 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.226174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 294/567] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:26.227458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ce979-3bd9-4a4a-b1e5-087ed90a2ede None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.227653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ce979-3bd9-4a4a-b1e5-087ed90a2ede None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.227989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7ce979-3bd9-4a4a-b1e5-087ed90a2ede None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.248085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-cd7ce979-3bd9-4a4a-b1e5-087ed90a2ede None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:26.249334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cd7ce979-3bd9-4a4a-b1e5-087ed90a2ede None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:26.272020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.297094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63552399-2b22-4be8-a1ba-e4cf2ce2df98 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.323897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 295/568] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:26.325321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ce979-3bd9-4a4a-b1e5-087ed90a2ede None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.348707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8fc8-ce92-471c-a8bb-7f2635e53982 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.349320 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8fc8-ce92-471c-a8bb-7f2635e53982 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.349673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8fc8-ce92-471c-a8bb-7f2635e53982 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.350361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-958e8fc8-ce92-471c-a8bb-7f2635e53982 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.351090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8fc8-ce92-471c-a8bb-7f2635e53982 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:26.351393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8fc8-ce92-471c-a8bb-7f2635e53982 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.352821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-958e8fc8-ce92-471c-a8bb-7f2635e53982 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.364695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 274/569] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:26.365537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8fc8-ce92-471c-a8bb-7f2635e53982 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.381727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.440278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7ce979-3bd9-4a4a-b1e5-087ed90a2ede None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.477383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-178267e5-2796-43dc-b101-d78834b66daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.478084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178267e5-2796-43dc-b101-d78834b66daf None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.478707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178267e5-2796-43dc-b101-d78834b66daf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.478707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-178267e5-2796-43dc-b101-d78834b66daf None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.479343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178267e5-2796-43dc-b101-d78834b66daf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:26.479650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178267e5-2796-43dc-b101-d78834b66daf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.481158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-178267e5-2796-43dc-b101-d78834b66daf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.494320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 296/570] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:26.495609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2247a-d2f3-4c43-8668-a91cdc10ae63 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.495900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2247a-d2f3-4c43-8668-a91cdc10ae63 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.496731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b2247a-d2f3-4c43-8668-a91cdc10ae63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.498357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-958e8fc8-ce92-471c-a8bb-7f2635e53982 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.517746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c2b2247a-d2f3-4c43-8668-a91cdc10ae63 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:26.519061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c2b2247a-d2f3-4c43-8668-a91cdc10ae63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:26.544576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e531e0-8bed-40ce-bf85-871303460fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.545249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e531e0-8bed-40ce-bf85-871303460fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.546023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e531e0-8bed-40ce-bf85-871303460fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.546023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92e531e0-8bed-40ce-bf85-871303460fc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.546560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e531e0-8bed-40ce-bf85-871303460fc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:26.546869 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e531e0-8bed-40ce-bf85-871303460fc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.548285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92e531e0-8bed-40ce-bf85-871303460fc5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.558568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.562977 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 275/571] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:26.565027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e531e0-8bed-40ce-bf85-871303460fc5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.596918 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 297/572] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:26.597816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2247a-d2f3-4c43-8668-a91cdc10ae63 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.647040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.677322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92e531e0-8bed-40ce-bf85-871303460fc5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.713344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15f0d7-0dd2-401c-9619-9b894014989c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.714184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15f0d7-0dd2-401c-9619-9b894014989c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.714469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15f0d7-0dd2-401c-9619-9b894014989c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.714792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c15f0d7-0dd2-401c-9619-9b894014989c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.725843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b2247a-d2f3-4c43-8668-a91cdc10ae63 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.727073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c15f0d7-0dd2-401c-9619-9b894014989c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-3c15f0d7-0dd2-401c-9619-9b894014989c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:26.735627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:26.737193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 276/573] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:26.739041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15f0d7-0dd2-401c-9619-9b894014989c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.772326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ad411-74a7-42dd-9388-0c0f3f686ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.773174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ad411-74a7-42dd-9388-0c0f3f686ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.773532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ad411-74a7-42dd-9388-0c0f3f686ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.773857 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-457ad411-74a7-42dd-9388-0c0f3f686ac8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.782509 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ad411-74a7-42dd-9388-0c0f3f686ac8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.784182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-457ad411-74a7-42dd-9388-0c0f3f686ac8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-457ad411-74a7-42dd-9388-0c0f3f686ac8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:26.799937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:26.801035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:26.801995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:26.801995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:26.801995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:26.801995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:26.801995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:26.801995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:26.801995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:26.801995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:26.801995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 298/574] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:26.801995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ad411-74a7-42dd-9388-0c0f3f686ac8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.807513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.854102 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c15f0d7-0dd2-401c-9619-9b894014989c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.889579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57dc8e-dd67-4f3d-ae53-8014d4b6536d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.890385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57dc8e-dd67-4f3d-ae53-8014d4b6536d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.890665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57dc8e-dd67-4f3d-ae53-8014d4b6536d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.890934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc57dc8e-dd67-4f3d-ae53-8014d4b6536d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.900119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57dc8e-dd67-4f3d-ae53-8014d4b6536d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.901725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc57dc8e-dd67-4f3d-ae53-8014d4b6536d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-cc57dc8e-dd67-4f3d-ae53-8014d4b6536d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:26.917206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:26.918438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 277/575] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:26.919396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57dc8e-dd67-4f3d-ae53-8014d4b6536d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:26.928184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:26.931263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-457ad411-74a7-42dd-9388-0c0f3f686ac8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:26.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ed460-68f9-48d5-9573-fa0e1d5d26b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:26.972761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ed460-68f9-48d5-9573-fa0e1d5d26b4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:26.973043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ed460-68f9-48d5-9573-fa0e1d5d26b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:26.973586 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa8ed460-68f9-48d5-9573-fa0e1d5d26b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:26.980318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ed460-68f9-48d5-9573-fa0e1d5d26b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:26.983310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa8ed460-68f9-48d5-9573-fa0e1d5d26b4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-aa8ed460-68f9-48d5-9573-fa0e1d5d26b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:26.998847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:26.999922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:27.001040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:27.001040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:27.001040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:27.001040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:27.001040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:27.001040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:27.001040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:27.001040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:27.001040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 299/576] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:27.001040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ed460-68f9-48d5-9573-fa0e1d5d26b4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.040198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc57dc8e-dd67-4f3d-ae53-8014d4b6536d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.062380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:27.077314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388539b-1216-44db-90cb-a92a39ec6a0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.078141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388539b-1216-44db-90cb-a92a39ec6a0f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.078422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388539b-1216-44db-90cb-a92a39ec6a0f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.078693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8388539b-1216-44db-90cb-a92a39ec6a0f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:27.088671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8388539b-1216-44db-90cb-a92a39ec6a0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8388539b-1216-44db-90cb-a92a39ec6a0f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:27.096411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:27.097543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 278/577] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:27.098672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388539b-1216-44db-90cb-a92a39ec6a0f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.116217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa8ed460-68f9-48d5-9573-fa0e1d5d26b4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.150598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01be27-9333-45b7-bccc-c9af7e7be3cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.151174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01be27-9333-45b7-bccc-c9af7e7be3cb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.151455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01be27-9333-45b7-bccc-c9af7e7be3cb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.151729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a01be27-9333-45b7-bccc-c9af7e7be3cb None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:27.152488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01be27-9333-45b7-bccc-c9af7e7be3cb None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:27.152946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01be27-9333-45b7-bccc-c9af7e7be3cb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:27.154595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a01be27-9333-45b7-bccc-c9af7e7be3cb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:27.162594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 300/578] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:27.163376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01be27-9333-45b7-bccc-c9af7e7be3cb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.189896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:27.204680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8388539b-1216-44db-90cb-a92a39ec6a0f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.239924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dccb6b-e71f-44fd-94ac-13abfedb4cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.240585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dccb6b-e71f-44fd-94ac-13abfedb4cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.240874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dccb6b-e71f-44fd-94ac-13abfedb4cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.241159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dccb6b-e71f-44fd-94ac-13abfedb4cf5 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:27.241980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dccb6b-e71f-44fd-94ac-13abfedb4cf5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:27.242288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dccb6b-e71f-44fd-94ac-13abfedb4cf5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:27.243736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dccb6b-e71f-44fd-94ac-13abfedb4cf5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:27.256373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 279/579] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:27.257199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dccb6b-e71f-44fd-94ac-13abfedb4cf5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.273332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a01be27-9333-45b7-bccc-c9af7e7be3cb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.300559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:27.308207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a238d-d0b0-4d66-aad4-24a255ed0a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.309442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a238d-d0b0-4d66-aad4-24a255ed0a2b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.309751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a238d-d0b0-4d66-aad4-24a255ed0a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.310140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880a238d-d0b0-4d66-aad4-24a255ed0a2b None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:27.310976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a238d-d0b0-4d66-aad4-24a255ed0a2b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:27.311287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a238d-d0b0-4d66-aad4-24a255ed0a2b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:27.313160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-880a238d-d0b0-4d66-aad4-24a255ed0a2b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:27.332848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 301/580] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Jul 20 19:10:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:27.333732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a238d-d0b0-4d66-aad4-24a255ed0a2b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.365922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dccb6b-e71f-44fd-94ac-13abfedb4cf5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.412785 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009dffe-3c21-476d-91ef-9a1699a82011 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.413368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009dffe-3c21-476d-91ef-9a1699a82011 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.413681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009dffe-3c21-476d-91ef-9a1699a82011 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.413962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4009dffe-3c21-476d-91ef-9a1699a82011 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:27.414751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009dffe-3c21-476d-91ef-9a1699a82011 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:27.415069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009dffe-3c21-476d-91ef-9a1699a82011 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:27.416529 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4009dffe-3c21-476d-91ef-9a1699a82011 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:27.428565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 280/581] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:27.430192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009dffe-3c21-476d-91ef-9a1699a82011 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.435012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:27.458469 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-880a238d-d0b0-4d66-aad4-24a255ed0a2b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.496112 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca3709-484a-407f-8466-18985a488568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.497339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca3709-484a-407f-8466-18985a488568 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.497755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca3709-484a-407f-8466-18985a488568 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.497969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ca3709-484a-407f-8466-18985a488568 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:27.498717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca3709-484a-407f-8466-18985a488568 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:27.499024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca3709-484a-407f-8466-18985a488568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:27.500396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ca3709-484a-407f-8466-18985a488568 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:27.513422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 302/582] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:27.514933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca3709-484a-407f-8466-18985a488568 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.539640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:27.542976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4009dffe-3c21-476d-91ef-9a1699a82011 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.579001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af0276-2126-4e0e-9220-62c66c60d08b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.579792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af0276-2126-4e0e-9220-62c66c60d08b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.580077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af0276-2126-4e0e-9220-62c66c60d08b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.580350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5af0276-2126-4e0e-9220-62c66c60d08b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:27.591220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af0276-2126-4e0e-9220-62c66c60d08b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:27.592673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5af0276-2126-4e0e-9220-62c66c60d08b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-b5af0276-2126-4e0e-9220-62c66c60d08b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:27.610597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:27.611712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:27.612640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:27.612640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:27.612640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:27.612640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:27.612640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 281/583] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:27.612640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af0276-2126-4e0e-9220-62c66c60d08b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ca3709-484a-407f-8466-18985a488568 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.664218 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae9f94-bd14-4534-93de-54cdef82f128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.664790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae9f94-bd14-4534-93de-54cdef82f128 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.665072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae9f94-bd14-4534-93de-54cdef82f128 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.665356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ae9f94-bd14-4534-93de-54cdef82f128 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:27.666215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae9f94-bd14-4534-93de-54cdef82f128 None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=9d8387328e4145db80782f2d54c0966d) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:27.666549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae9f94-bd14-4534-93de-54cdef82f128 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:27.667971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ae9f94-bd14-4534-93de-54cdef82f128 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:27.673574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 303/584] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:27 2022] GET /identity/v3/role_assignments?user.id=9d8387328e4145db80782f2d54c0966d&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 259 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:27.674329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae9f94-bd14-4534-93de-54cdef82f128 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.684275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:27.729312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5af0276-2126-4e0e-9220-62c66c60d08b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.769807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d04cf0-0976-4ea5-9b50-8163c9ccd0a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.771473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d04cf0-0976-4ea5-9b50-8163c9ccd0a3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.771860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d04cf0-0976-4ea5-9b50-8163c9ccd0a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.772237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d04cf0-0976-4ea5-9b50-8163c9ccd0a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:27.787922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d04cf0-0976-4ea5-9b50-8163c9ccd0a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:27.790371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d04cf0-0976-4ea5-9b50-8163c9ccd0a3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:27.795808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ae9f94-bd14-4534-93de-54cdef82f128 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-31d04cf0-0976-4ea5-9b50-8163c9ccd0a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:27.810980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:27.812171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:27.813278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:27.813278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:27.813278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:27.813278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:27.814032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 282/585] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:27.814858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d04cf0-0976-4ea5-9b50-8163c9ccd0a3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.819491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:27.838979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496fc11-44c5-476f-a0b1-82ce3681e842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.839874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496fc11-44c5-476f-a0b1-82ce3681e842 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.840876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496fc11-44c5-476f-a0b1-82ce3681e842 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.841160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c496fc11-44c5-476f-a0b1-82ce3681e842 None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/3e46ec9c31094d03b6a413e09bc9f699/roles/5b019d81d9cf4a16b6c6d1caafec9081` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:27.862855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496fc11-44c5-476f-a0b1-82ce3681e842 None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=3e46ec9c31094d03b6a413e09bc9f699, role_id=5b019d81d9cf4a16b6c6d1caafec9081)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:27.864460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c496fc11-44c5-476f-a0b1-82ce3681e842 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:27.879317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 304/586] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:27 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/3e46ec9c31094d03b6a413e09bc9f699/roles/5b019d81d9cf4a16b6c6d1caafec9081 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:27.882447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496fc11-44c5-476f-a0b1-82ce3681e842 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:27.942362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d04cf0-0976-4ea5-9b50-8163c9ccd0a3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:27.994605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99b60e-ff96-40bb-8838-edbaf2f825a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:27.995240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:27.996724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99b60e-ff96-40bb-8838-edbaf2f825a8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:27.997482 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99b60e-ff96-40bb-8838-edbaf2f825a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:27.997953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d99b60e-ff96-40bb-8838-edbaf2f825a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.015314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99b60e-ff96-40bb-8838-edbaf2f825a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.017883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d99b60e-ff96-40bb-8838-edbaf2f825a8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.022733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c496fc11-44c5-476f-a0b1-82ce3681e842 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1d99b60e-ff96-40bb-8838-edbaf2f825a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:28.039577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:28.040666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:28.041799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:28.041799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:28.041799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:28.041799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:28.045887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 283/587] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:28.046993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99b60e-ff96-40bb-8838-edbaf2f825a8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.073156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88906c-5a88-45fa-83f3-251793c95368 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.074133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88906c-5a88-45fa-83f3-251793c95368 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.075486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88906c-5a88-45fa-83f3-251793c95368 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.075771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e88906c-5a88-45fa-83f3-251793c95368 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.076500 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88906c-5a88-45fa-83f3-251793c95368 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:28.076805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88906c-5a88-45fa-83f3-251793c95368 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.078334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e88906c-5a88-45fa-83f3-251793c95368 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.092489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 305/588] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:28.097497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88906c-5a88-45fa-83f3-251793c95368 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.126320 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.166592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d99b60e-ff96-40bb-8838-edbaf2f825a8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.203956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ee044-5039-4db4-b2cf-b87b2fb50cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.204693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ee044-5039-4db4-b2cf-b87b2fb50cfc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.204986 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ee044-5039-4db4-b2cf-b87b2fb50cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.205265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ee044-5039-4db4-b2cf-b87b2fb50cfc None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.206020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ee044-5039-4db4-b2cf-b87b2fb50cfc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:28.206330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ee044-5039-4db4-b2cf-b87b2fb50cfc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.207832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ee044-5039-4db4-b2cf-b87b2fb50cfc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.220992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 284/589] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:28.221432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e88906c-5a88-45fa-83f3-251793c95368 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.221875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ee044-5039-4db4-b2cf-b87b2fb50cfc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.242533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.266678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8faf67-80a9-4145-9d28-4648d44436ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.267354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8faf67-80a9-4145-9d28-4648d44436ab None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.267639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8faf67-80a9-4145-9d28-4648d44436ab None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.267917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8faf67-80a9-4145-9d28-4648d44436ab None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.268648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8faf67-80a9-4145-9d28-4648d44436ab None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:28.268955 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8faf67-80a9-4145-9d28-4648d44436ab None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.270362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d8faf67-80a9-4145-9d28-4648d44436ab None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.286086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 306/590] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:28.288198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8faf67-80a9-4145-9d28-4648d44436ab None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.354786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ee044-5039-4db4-b2cf-b87b2fb50cfc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.385907 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.405291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102e503-768b-4de0-8487-5dcb83c0606b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.406404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102e503-768b-4de0-8487-5dcb83c0606b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.406687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102e503-768b-4de0-8487-5dcb83c0606b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.406956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e102e503-768b-4de0-8487-5dcb83c0606b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.417546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102e503-768b-4de0-8487-5dcb83c0606b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.419721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e102e503-768b-4de0-8487-5dcb83c0606b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.420083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8faf67-80a9-4145-9d28-4648d44436ab None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e102e503-768b-4de0-8487-5dcb83c0606b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:28.444443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:28.445671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:28.446707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:28.446707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:28.446707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:28.446707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:28.446707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:28.446707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:28.446707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:28.446707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:28.446707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:10:28.446707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:28.447557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 285/591] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:28.448402 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102e503-768b-4de0-8487-5dcb83c0606b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.465907 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58fc3b-09a2-4a7d-af18-9bb59dd655d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.466800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58fc3b-09a2-4a7d-af18-9bb59dd655d6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.467086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58fc3b-09a2-4a7d-af18-9bb59dd655d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.467364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e58fc3b-09a2-4a7d-af18-9bb59dd655d6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.476868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58fc3b-09a2-4a7d-af18-9bb59dd655d6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.478505 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e58fc3b-09a2-4a7d-af18-9bb59dd655d6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2e58fc3b-09a2-4a7d-af18-9bb59dd655d6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:28.498133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:28.499520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:28.500425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:28.500425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:28.500425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:28.500425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:28.500425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:28.500425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:28.500425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:28.500425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:28.500425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 307/592] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:28.500935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58fc3b-09a2-4a7d-af18-9bb59dd655d6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.547692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.572159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e102e503-768b-4de0-8487-5dcb83c0606b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.609817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1c252-39c3-4e10-931d-50b8c0d2de50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.610956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1c252-39c3-4e10-931d-50b8c0d2de50 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.611432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1c252-39c3-4e10-931d-50b8c0d2de50 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.611920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f1c252-39c3-4e10-931d-50b8c0d2de50 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.620140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1c252-39c3-4e10-931d-50b8c0d2de50 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.621980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f1c252-39c3-4e10-931d-50b8c0d2de50 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.632439 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e58fc3b-09a2-4a7d-af18-9bb59dd655d6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f5f1c252-39c3-4e10-931d-50b8c0d2de50 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:28.638575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:28.639516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:28.640481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:28.640481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:28.640481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:28.640481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:28.640481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:28.640481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:28.640481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:28.640481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:28.640481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:10:28.640481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:28.643189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 286/593] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/users/neutron => generated 85 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:28.644181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1c252-39c3-4e10-931d-50b8c0d2de50 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.682188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.686386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a713f-1b72-4847-b063-bd72f186312e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.687068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a713f-1b72-4847-b063-bd72f186312e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.687353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a713f-1b72-4847-b063-bd72f186312e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.687630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967a713f-1b72-4847-b063-bd72f186312e None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.688716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a713f-1b72-4847-b063-bd72f186312e None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:28.689196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a713f-1b72-4847-b063-bd72f186312e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.690689 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967a713f-1b72-4847-b063-bd72f186312e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.717490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 308/594] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:28.718558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a713f-1b72-4847-b063-bd72f186312e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.761721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f1c252-39c3-4e10-931d-50b8c0d2de50 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.798038 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb28687-bb07-4164-9029-ec260b297f0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.798720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb28687-bb07-4164-9029-ec260b297f0e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.799007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb28687-bb07-4164-9029-ec260b297f0e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.799287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb28687-bb07-4164-9029-ec260b297f0e None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.800093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb28687-bb07-4164-9029-ec260b297f0e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:28.800408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb28687-bb07-4164-9029-ec260b297f0e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.801950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb28687-bb07-4164-9029-ec260b297f0e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.815837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 287/595] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:28.816735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb28687-bb07-4164-9029-ec260b297f0e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.847557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.857892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967a713f-1b72-4847-b063-bd72f186312e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.901103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b20578-0aa3-473c-997d-9ca786c21364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.901965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20578-0aa3-473c-997d-9ca786c21364 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.902250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20578-0aa3-473c-997d-9ca786c21364 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.902576 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b20578-0aa3-473c-997d-9ca786c21364 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:28.904284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b20578-0aa3-473c-997d-9ca786c21364 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:28.904607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b20578-0aa3-473c-997d-9ca786c21364 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:28.906087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b20578-0aa3-473c-997d-9ca786c21364 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:28.928460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 309/596] 10.222.0.69 () {56 vars in 1139 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:28.929289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b20578-0aa3-473c-997d-9ca786c21364 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:28.935185 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:28.942770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb28687-bb07-4164-9029-ec260b297f0e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:28.986787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b6029-b40e-4af3-a803-b18b6d38cd0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:28.988102 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b6029-b40e-4af3-a803-b18b6d38cd0d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:28.988934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b6029-b40e-4af3-a803-b18b6d38cd0d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:28.989717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b6029-b40e-4af3-a803-b18b6d38cd0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.004907 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b6029-b40e-4af3-a803-b18b6d38cd0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.007578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b6029-b40e-4af3-a803-b18b6d38cd0d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-d15b6029-b40e-4af3-a803-b18b6d38cd0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:29.028671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:29.029666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:29.031161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:29.031161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:29.031161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:29.031161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:29.031161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 288/597] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:29.031161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b6029-b40e-4af3-a803-b18b6d38cd0d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.055659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b20578-0aa3-473c-997d-9ca786c21364 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.096583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b38f2c-97d3-4c56-be72-17ce78e7ddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.097918 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b38f2c-97d3-4c56-be72-17ce78e7ddb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.098675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b38f2c-97d3-4c56-be72-17ce78e7ddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.099152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b38f2c-97d3-4c56-be72-17ce78e7ddb9 None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/36955d7d4505457696ab6177adb999a9/roles/5b019d81d9cf4a16b6c6d1caafec9081` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.117858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b38f2c-97d3-4c56-be72-17ce78e7ddb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=36955d7d4505457696ab6177adb999a9, role_id=5b019d81d9cf4a16b6c6d1caafec9081)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.119641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61b38f2c-97d3-4c56-be72-17ce78e7ddb9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.137953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b6029-b40e-4af3-a803-b18b6d38cd0d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.138966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 310/598] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:28 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/36955d7d4505457696ab6177adb999a9/roles/5b019d81d9cf4a16b6c6d1caafec9081 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:29.141365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b38f2c-97d3-4c56-be72-17ce78e7ddb9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.144697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:29.210681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42f55a-0370-4c9a-ae3e-534f0da04782 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.211546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42f55a-0370-4c9a-ae3e-534f0da04782 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.211880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42f55a-0370-4c9a-ae3e-534f0da04782 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.212202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e42f55a-0370-4c9a-ae3e-534f0da04782 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.237236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42f55a-0370-4c9a-ae3e-534f0da04782 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.238083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e42f55a-0370-4c9a-ae3e-534f0da04782 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.260142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-9e42f55a-0370-4c9a-ae3e-534f0da04782 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:29.278688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:29.279674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:29.280574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:29.280574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:29.280574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:29.280574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:29.283256 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61b38f2c-97d3-4c56-be72-17ce78e7ddb9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.285975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 289/599] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:29.291387 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42f55a-0370-4c9a-ae3e-534f0da04782 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.335910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cdc91-5fa2-4a8a-9633-2676fb9da6cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.336742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cdc91-5fa2-4a8a-9633-2676fb9da6cf None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.337026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cdc91-5fa2-4a8a-9633-2676fb9da6cf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.337345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5cdc91-5fa2-4a8a-9633-2676fb9da6cf None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.338111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cdc91-5fa2-4a8a-9633-2676fb9da6cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:29.338415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cdc91-5fa2-4a8a-9633-2676fb9da6cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.340039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5cdc91-5fa2-4a8a-9633-2676fb9da6cf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.365579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 311/600] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:29.372779 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cdc91-5fa2-4a8a-9633-2676fb9da6cf None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.457994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:29.485111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e42f55a-0370-4c9a-ae3e-534f0da04782 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.495244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5cdc91-5fa2-4a8a-9633-2676fb9da6cf None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.532205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec0dc0-5c2e-4da9-bf03-784c96320720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.532995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec0dc0-5c2e-4da9-bf03-784c96320720 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.533285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec0dc0-5c2e-4da9-bf03-784c96320720 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.533589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec0dc0-5c2e-4da9-bf03-784c96320720 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.540763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec0dc0-5c2e-4da9-bf03-784c96320720 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.542240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec0dc0-5c2e-4da9-bf03-784c96320720 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-eeec0dc0-5c2e-4da9-bf03-784c96320720 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:29.557155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:29.558141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:29.560070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:29.560070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:29.560070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:29.560070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:29.560070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:29.560070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:29.560070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:29.560070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:29.560070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 312/601] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:29.560559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246ab11-8a15-4dd7-9154-a82bf47b8fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.560559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ab11-8a15-4dd7-9154-a82bf47b8fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.560704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ab11-8a15-4dd7-9154-a82bf47b8fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.565187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c246ab11-8a15-4dd7-9154-a82bf47b8fd7 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.565187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246ab11-8a15-4dd7-9154-a82bf47b8fd7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:29.565187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246ab11-8a15-4dd7-9154-a82bf47b8fd7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.565187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c246ab11-8a15-4dd7-9154-a82bf47b8fd7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.570470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec0dc0-5c2e-4da9-bf03-784c96320720 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.581625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 290/602] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:29.587534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246ab11-8a15-4dd7-9154-a82bf47b8fd7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.711238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec0dc0-5c2e-4da9-bf03-784c96320720 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.718865 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:29.751678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:29.755985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474317c2-9442-4a9c-a0b6-5c02c107d3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.756658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474317c2-9442-4a9c-a0b6-5c02c107d3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.756938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474317c2-9442-4a9c-a0b6-5c02c107d3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.757209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474317c2-9442-4a9c-a0b6-5c02c107d3c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.758034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474317c2-9442-4a9c-a0b6-5c02c107d3c1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:29.758341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474317c2-9442-4a9c-a0b6-5c02c107d3c1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.759780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474317c2-9442-4a9c-a0b6-5c02c107d3c1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.791567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 313/603] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:29.792505 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474317c2-9442-4a9c-a0b6-5c02c107d3c1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.819099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c246ab11-8a15-4dd7-9154-a82bf47b8fd7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.875281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2892f5-8cb3-43e9-8037-ee9a932b3c79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.876802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2892f5-8cb3-43e9-8037-ee9a932b3c79 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.877267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2892f5-8cb3-43e9-8037-ee9a932b3c79 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.877704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2892f5-8cb3-43e9-8037-ee9a932b3c79 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.885363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2892f5-8cb3-43e9-8037-ee9a932b3c79 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.887465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2892f5-8cb3-43e9-8037-ee9a932b3c79 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-4c2892f5-8cb3-43e9-8037-ee9a932b3c79 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:29.902444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:29.903904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:29.905647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:29.905647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:29.905647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:29.905647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:29.905647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:29.905647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:29.905647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:29.905647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:29.909000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 291/604] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 317 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:29.910642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f16513-ea30-41b8-85ad-69d4db72ff2e None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.910934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f16513-ea30-41b8-85ad-69d4db72ff2e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.911238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f16513-ea30-41b8-85ad-69d4db72ff2e None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.912857 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 292/605] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:29.917064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474317c2-9442-4a9c-a0b6-5c02c107d3c1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:29.918607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f16513-ea30-41b8-85ad-69d4db72ff2e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:29.954477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e342bdf8-4ab0-41a7-9c03-73b127997285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:29.955361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342bdf8-4ab0-41a7-9c03-73b127997285 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.955642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342bdf8-4ab0-41a7-9c03-73b127997285 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.955909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e342bdf8-4ab0-41a7-9c03-73b127997285 None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/c94d9030f61a4e1f8b3d75ef8925bdd0/roles/5b019d81d9cf4a16b6c6d1caafec9081` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:29.971500 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342bdf8-4ab0-41a7-9c03-73b127997285 None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=c94d9030f61a4e1f8b3d75ef8925bdd0, role_id=5b019d81d9cf4a16b6c6d1caafec9081)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:29.973053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e342bdf8-4ab0-41a7-9c03-73b127997285 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:29.990352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 314/606] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:29 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/c94d9030f61a4e1f8b3d75ef8925bdd0/roles/5b019d81d9cf4a16b6c6d1caafec9081 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:29.999030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd22ad-1dbd-43fa-91c5-9403cc0abeea None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:29.999332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd22ad-1dbd-43fa-91c5-9403cc0abeea None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:29.999742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd22ad-1dbd-43fa-91c5-9403cc0abeea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.021348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b1bd22ad-1dbd-43fa-91c5-9403cc0abeea None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:30.022638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b1bd22ad-1dbd-43fa-91c5-9403cc0abeea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:30.055605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.069145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f16513-ea30-41b8-85ad-69d4db72ff2e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.094516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.112240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3f8a0-872b-44a5-a4c2-2de27ca19739 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.112925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3f8a0-872b-44a5-a4c2-2de27ca19739 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.113204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3f8a0-872b-44a5-a4c2-2de27ca19739 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.113474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3f8a0-872b-44a5-a4c2-2de27ca19739 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.114371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3f8a0-872b-44a5-a4c2-2de27ca19739 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:30.115416 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3f8a0-872b-44a5-a4c2-2de27ca19739 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:30.117132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3f8a0-872b-44a5-a4c2-2de27ca19739 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:30.133810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 293/607] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:30.144759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3f8a0-872b-44a5-a4c2-2de27ca19739 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.158276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 315/608] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:30.200652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd22ad-1dbd-43fa-91c5-9403cc0abeea None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.270834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3f8a0-872b-44a5-a4c2-2de27ca19739 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.325021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f29e741d-4426-4bf4-9a86-ac8da1eb3ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.325979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e741d-4426-4bf4-9a86-ac8da1eb3ae5 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.326263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e741d-4426-4bf4-9a86-ac8da1eb3ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.326541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f29e741d-4426-4bf4-9a86-ac8da1eb3ae5 None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/3a1a88089d384964bd9774eda4c5d8a9/roles/5b019d81d9cf4a16b6c6d1caafec9081` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.332659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd22ad-1dbd-43fa-91c5-9403cc0abeea None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.342891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e741d-4426-4bf4-9a86-ac8da1eb3ae5 None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=3a1a88089d384964bd9774eda4c5d8a9, role_id=5b019d81d9cf4a16b6c6d1caafec9081)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:30.344008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f29e741d-4426-4bf4-9a86-ac8da1eb3ae5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:30.365571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 294/609] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:30 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/3a1a88089d384964bd9774eda4c5d8a9/roles/5b019d81d9cf4a16b6c6d1caafec9081 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:30.365571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9573c05-1cee-4337-825a-bb63592097d7 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.365571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9573c05-1cee-4337-825a-bb63592097d7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.365571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9573c05-1cee-4337-825a-bb63592097d7 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.365571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 295/610] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:30.372410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97661d0-040e-473a-8e44-620c002588a2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.372984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97661d0-040e-473a-8e44-620c002588a2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.373416 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97661d0-040e-473a-8e44-620c002588a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.399099 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5176394-927b-4310-a5b2-8d249fc8930d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.400210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5176394-927b-4310-a5b2-8d249fc8930d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.400645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5176394-927b-4310-a5b2-8d249fc8930d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.401072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5176394-927b-4310-a5b2-8d249fc8930d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.413987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b97661d0-040e-473a-8e44-620c002588a2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:30.415167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5176394-927b-4310-a5b2-8d249fc8930d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:30.415331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b97661d0-040e-473a-8e44-620c002588a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b5176394-927b-4310-a5b2-8d249fc8930d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:30.425160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:30.426419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:30.427491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:30.430671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 316/611] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:30.437433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5176394-927b-4310-a5b2-8d249fc8930d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.463186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.500693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.524708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 296/612] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:30.568401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97661d0-040e-473a-8e44-620c002588a2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.612285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5176394-927b-4310-a5b2-8d249fc8930d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.651771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b5fbd-991e-460b-a06b-822b8de2bfc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.653146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b5fbd-991e-460b-a06b-822b8de2bfc9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.653849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b5fbd-991e-460b-a06b-822b8de2bfc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.654137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6b5fbd-991e-460b-a06b-822b8de2bfc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.655059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b5fbd-991e-460b-a06b-822b8de2bfc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:30.655373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b5fbd-991e-460b-a06b-822b8de2bfc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:30.657117 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6b5fbd-991e-460b-a06b-822b8de2bfc9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:30.671764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 317/613] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:30.677171 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b5fbd-991e-460b-a06b-822b8de2bfc9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.710473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97661d0-040e-473a-8e44-620c002588a2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.790223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685871a-ac9f-424b-9477-c5caad56e754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.791047 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685871a-ac9f-424b-9477-c5caad56e754 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.791157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.791332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685871a-ac9f-424b-9477-c5caad56e754 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.791610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4685871a-ac9f-424b-9477-c5caad56e754 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.804165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6b5fbd-991e-460b-a06b-822b8de2bfc9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:30.822396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4685871a-ac9f-424b-9477-c5caad56e754 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-4685871a-ac9f-424b-9477-c5caad56e754 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:30.834741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:30.835699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:30.836553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 297/614] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 268 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:30.843020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685871a-ac9f-424b-9477-c5caad56e754 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.850473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95447980-7522-412d-b623-db62fa14b499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:30.851337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95447980-7522-412d-b623-db62fa14b499 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:30.851624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95447980-7522-412d-b623-db62fa14b499 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:30.851904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95447980-7522-412d-b623-db62fa14b499 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:30.873616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95447980-7522-412d-b623-db62fa14b499 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:30.876580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95447980-7522-412d-b623-db62fa14b499 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:30.893883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-95447980-7522-412d-b623-db62fa14b499 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:30.917778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:30.919032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:30.919948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:30.919948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:30.919948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:30.919948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:30.919948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 318/615] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:30.926503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95447980-7522-412d-b623-db62fa14b499 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:30.985404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4685871a-ac9f-424b-9477-c5caad56e754 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.023236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1c180-eb4f-4324-8fdf-4f1421ef63d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.023972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1c180-eb4f-4324-8fdf-4f1421ef63d3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.024264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1c180-eb4f-4324-8fdf-4f1421ef63d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.024544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a1c180-eb4f-4324-8fdf-4f1421ef63d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.025295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1c180-eb4f-4324-8fdf-4f1421ef63d3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:31.025633 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1c180-eb4f-4324-8fdf-4f1421ef63d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.027141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a1c180-eb4f-4324-8fdf-4f1421ef63d3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.040283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 298/616] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:31.044112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1c180-eb4f-4324-8fdf-4f1421ef63d3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.122512 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95447980-7522-412d-b623-db62fa14b499 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.162471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9650b0-1dac-416b-9a95-225a7e7de40c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.163140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9650b0-1dac-416b-9a95-225a7e7de40c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.163430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9650b0-1dac-416b-9a95-225a7e7de40c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.163713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b9650b0-1dac-416b-9a95-225a7e7de40c None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.164599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9650b0-1dac-416b-9a95-225a7e7de40c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:31.164920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9650b0-1dac-416b-9a95-225a7e7de40c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.166383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b9650b0-1dac-416b-9a95-225a7e7de40c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.170259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.181758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 319/617] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:31.187751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9650b0-1dac-416b-9a95-225a7e7de40c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.249670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.285228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a1c180-eb4f-4324-8fdf-4f1421ef63d3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.351506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b9650b0-1dac-416b-9a95-225a7e7de40c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.384837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bf83c-8ba1-4358-8ed3-30d9f98048fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.385679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bf83c-8ba1-4358-8ed3-30d9f98048fa None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.385969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bf83c-8ba1-4358-8ed3-30d9f98048fa None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.386245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471bf83c-8ba1-4358-8ed3-30d9f98048fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.413918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bf83c-8ba1-4358-8ed3-30d9f98048fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.418471 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471bf83c-8ba1-4358-8ed3-30d9f98048fa None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.426315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1915dd8-c37c-4903-84f8-100ac404a91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.427126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1915dd8-c37c-4903-84f8-100ac404a91f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.427407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1915dd8-c37c-4903-84f8-100ac404a91f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.427680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1915dd8-c37c-4903-84f8-100ac404a91f None admin] PATH_INFO: `/v3/users/swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.446558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1915dd8-c37c-4903-84f8-100ac404a91f None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.447999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1915dd8-c37c-4903-84f8-100ac404a91f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-471bf83c-8ba1-4358-8ed3-30d9f98048fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:31.461096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:31.462042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:31.462941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:31.462941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:31.462941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:31.462941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:31.465703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 299/618] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 422 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:31.470995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bf83c-8ba1-4358-8ed3-30d9f98048fa None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f1915dd8-c37c-4903-84f8-100ac404a91f None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:31.479565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:31.481083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:31.482343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:31.482343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:31.482343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:31.482343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:31.482343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:31.482343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:31.482343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:31.482343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:31.482343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:31.482343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:31.487972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 320/619] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:31.490547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1915dd8-c37c-4903-84f8-100ac404a91f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.608725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.635410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471bf83c-8ba1-4358-8ed3-30d9f98048fa None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.653919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1915dd8-c37c-4903-84f8-100ac404a91f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.673104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d397eda5-2da2-4c48-a7ac-6fafa9bf0265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.674344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397eda5-2da2-4c48-a7ac-6fafa9bf0265 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.675006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397eda5-2da2-4c48-a7ac-6fafa9bf0265 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.675768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d397eda5-2da2-4c48-a7ac-6fafa9bf0265 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.676703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397eda5-2da2-4c48-a7ac-6fafa9bf0265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:31.677219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397eda5-2da2-4c48-a7ac-6fafa9bf0265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.679276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d397eda5-2da2-4c48-a7ac-6fafa9bf0265 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.681318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.700541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079fa84-cb82-4550-a2ab-9299a9c83db7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.700541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079fa84-cb82-4550-a2ab-9299a9c83db7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.700541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079fa84-cb82-4550-a2ab-9299a9c83db7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.700541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a079fa84-cb82-4550-a2ab-9299a9c83db7 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.701126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079fa84-cb82-4550-a2ab-9299a9c83db7 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:31.701126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079fa84-cb82-4550-a2ab-9299a9c83db7 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.701333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 300/620] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:31.701333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dfd44-ace6-4375-84f2-6a8fc6bede98 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.701571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a079fa84-cb82-4550-a2ab-9299a9c83db7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.701695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dfd44-ace6-4375-84f2-6a8fc6bede98 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.701695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094dfd44-ace6-4375-84f2-6a8fc6bede98 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.701893 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 301/621] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:31.706669 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e8e51-93e8-4bdb-a48e-cb9079b15623 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.706669 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e8e51-93e8-4bdb-a48e-cb9079b15623 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.706970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054e8e51-93e8-4bdb-a48e-cb9079b15623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.724757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 321/622] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:31.725820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079fa84-cb82-4550-a2ab-9299a9c83db7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.729784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-054e8e51-93e8-4bdb-a48e-cb9079b15623 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:31.731629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-054e8e51-93e8-4bdb-a48e-cb9079b15623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:31.812004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 302/623] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:31.813167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e8e51-93e8-4bdb-a48e-cb9079b15623 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.840484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a079fa84-cb82-4550-a2ab-9299a9c83db7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.857141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.881298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59137a29-3199-44a3-8694-d8fe5ebb1d97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.882391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59137a29-3199-44a3-8694-d8fe5ebb1d97 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.882817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59137a29-3199-44a3-8694-d8fe5ebb1d97 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.883181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59137a29-3199-44a3-8694-d8fe5ebb1d97 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.892196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59137a29-3199-44a3-8694-d8fe5ebb1d97 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.893811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59137a29-3199-44a3-8694-d8fe5ebb1d97 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-59137a29-3199-44a3-8694-d8fe5ebb1d97 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:31.910561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:31.911656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:31.912831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 322/624] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/users/glance => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:31.917899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59137a29-3199-44a3-8694-d8fe5ebb1d97 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:31.926257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054e8e51-93e8-4bdb-a48e-cb9079b15623 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:31.953574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:31.966098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768dfae-77b4-49f0-ac8a-97f9303cf30f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:31.966944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768dfae-77b4-49f0-ac8a-97f9303cf30f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:31.967225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768dfae-77b4-49f0-ac8a-97f9303cf30f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:31.967594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0768dfae-77b4-49f0-ac8a-97f9303cf30f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:31.979318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768dfae-77b4-49f0-ac8a-97f9303cf30f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:31.981587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0768dfae-77b4-49f0-ac8a-97f9303cf30f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-0768dfae-77b4-49f0-ac8a-97f9303cf30f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:32.000547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:32.001577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:32.002523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:32.002523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:32.002523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:32.002523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:32.006405 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 303/625] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:32.007332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768dfae-77b4-49f0-ac8a-97f9303cf30f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.045897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59137a29-3199-44a3-8694-d8fe5ebb1d97 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.085234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bc773-59e7-4cb0-84a7-7753a7d74b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.086470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bc773-59e7-4cb0-84a7-7753a7d74b8a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.086970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bc773-59e7-4cb0-84a7-7753a7d74b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.087463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46bc773-59e7-4cb0-84a7-7753a7d74b8a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.099294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46bc773-59e7-4cb0-84a7-7753a7d74b8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b46bc773-59e7-4cb0-84a7-7753a7d74b8a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:32.108717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:32.109753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:32.110611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:32.113621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 323/626] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:32.114650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bc773-59e7-4cb0-84a7-7753a7d74b8a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.115887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0768dfae-77b4-49f0-ac8a-97f9303cf30f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.155936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.158187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637bfe03-77f6-4d5f-9e64-b05db916b564 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.159101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bfe03-77f6-4d5f-9e64-b05db916b564 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.159555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bfe03-77f6-4d5f-9e64-b05db916b564 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.160002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637bfe03-77f6-4d5f-9e64-b05db916b564 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.161033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637bfe03-77f6-4d5f-9e64-b05db916b564 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:32.161500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637bfe03-77f6-4d5f-9e64-b05db916b564 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.163887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-637bfe03-77f6-4d5f-9e64-b05db916b564 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.185494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 304/627] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.187357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637bfe03-77f6-4d5f-9e64-b05db916b564 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.226319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.236927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46bc773-59e7-4cb0-84a7-7753a7d74b8a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.274593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a964ecf-810f-4b7d-aec0-afb74fbbb05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.275418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a964ecf-810f-4b7d-aec0-afb74fbbb05c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.275799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a964ecf-810f-4b7d-aec0-afb74fbbb05c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.276204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a964ecf-810f-4b7d-aec0-afb74fbbb05c None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.277373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a964ecf-810f-4b7d-aec0-afb74fbbb05c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:32.278411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a964ecf-810f-4b7d-aec0-afb74fbbb05c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.279812 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a964ecf-810f-4b7d-aec0-afb74fbbb05c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.298003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 324/628] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.298003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a964ecf-810f-4b7d-aec0-afb74fbbb05c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.306037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637bfe03-77f6-4d5f-9e64-b05db916b564 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.345594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73334741-ecad-4cb6-9f51-925736630bc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.346705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73334741-ecad-4cb6-9f51-925736630bc7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.347220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73334741-ecad-4cb6-9f51-925736630bc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.347708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73334741-ecad-4cb6-9f51-925736630bc7 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.348932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73334741-ecad-4cb6-9f51-925736630bc7 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:32.349471 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73334741-ecad-4cb6-9f51-925736630bc7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.351253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73334741-ecad-4cb6-9f51-925736630bc7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.361130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 305/629] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.362535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73334741-ecad-4cb6-9f51-925736630bc7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.408831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.416799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a964ecf-810f-4b7d-aec0-afb74fbbb05c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.453284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1d8270-f22d-4344-b4ba-5b131c763662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.453887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d8270-f22d-4344-b4ba-5b131c763662 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.454163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d8270-f22d-4344-b4ba-5b131c763662 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.454438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1d8270-f22d-4344-b4ba-5b131c763662 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.475396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.481921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73334741-ecad-4cb6-9f51-925736630bc7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.519569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6afe8d-e19f-4561-8a2b-34a03b5f1ccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.520458 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6afe8d-e19f-4561-8a2b-34a03b5f1ccf None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.520742 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6afe8d-e19f-4561-8a2b-34a03b5f1ccf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.521014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6afe8d-e19f-4561-8a2b-34a03b5f1ccf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.528703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6afe8d-e19f-4561-8a2b-34a03b5f1ccf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.530134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d8270-f22d-4344-b4ba-5b131c763662 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.530270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6afe8d-e19f-4561-8a2b-34a03b5f1ccf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.531553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1d8270-f22d-4344-b4ba-5b131c763662 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-0a6afe8d-e19f-4561-8a2b-34a03b5f1ccf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:32.545577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:32.546549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:32.547438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:32.547438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:32.547438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:32.547438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:32.547438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:32.547438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:32.547438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:32.547438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:32.547438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 306/630] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:32.547438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6afe8d-e19f-4561-8a2b-34a03b5f1ccf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.644719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.654656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 325/631] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:32.656839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfae0ee-b70a-40b4-9142-d533451965de None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.657352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfae0ee-b70a-40b4-9142-d533451965de None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.657879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfae0ee-b70a-40b4-9142-d533451965de None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.659532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 326/632] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:32.660476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfae0ee-b70a-40b4-9142-d533451965de None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.667658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6afe8d-e19f-4561-8a2b-34a03b5f1ccf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.708252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecaeb6-aeea-4169-b55b-bed2dc636970 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.708970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecaeb6-aeea-4169-b55b-bed2dc636970 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.709255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecaeb6-aeea-4169-b55b-bed2dc636970 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.709552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eecaeb6-aeea-4169-b55b-bed2dc636970 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.720028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecaeb6-aeea-4169-b55b-bed2dc636970 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.721657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eecaeb6-aeea-4169-b55b-bed2dc636970 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1eecaeb6-aeea-4169-b55b-bed2dc636970 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:32.741243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:32.742449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:32.743351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:32.743351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:32.743351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:32.743351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:32.748117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 307/633] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:32.748397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecaeb6-aeea-4169-b55b-bed2dc636970 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.759636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.777379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfae0ee-b70a-40b4-9142-d533451965de None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.812238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1e898-3429-4241-af69-dc16bfbd2107 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.812886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e898-3429-4241-af69-dc16bfbd2107 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.813165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e898-3429-4241-af69-dc16bfbd2107 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.813479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1e898-3429-4241-af69-dc16bfbd2107 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.814286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1e898-3429-4241-af69-dc16bfbd2107 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:32.814590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1e898-3429-4241-af69-dc16bfbd2107 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.816011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1e898-3429-4241-af69-dc16bfbd2107 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.827811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 327/634] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:32.829098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd7688-304c-4c0c-a34a-264004e98ab7 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.829382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd7688-304c-4c0c-a34a-264004e98ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.829683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edd7688-304c-4c0c-a34a-264004e98ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.845302 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6edd7688-304c-4c0c-a34a-264004e98ab7 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:32.846500 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6edd7688-304c-4c0c-a34a-264004e98ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:32.858472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eecaeb6-aeea-4169-b55b-bed2dc636970 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:32.893083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3977f-f25c-4178-a2d9-36b32a18e5e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:32.894044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3977f-f25c-4178-a2d9-36b32a18e5e4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.894425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3977f-f25c-4178-a2d9-36b32a18e5e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.894802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3977f-f25c-4178-a2d9-36b32a18e5e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.896240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:32.905492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3977f-f25c-4178-a2d9-36b32a18e5e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:32.906970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3977f-f25c-4178-a2d9-36b32a18e5e4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:32.913317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 328/635] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:32.914167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd7688-304c-4c0c-a34a-264004e98ab7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-09b3977f-f25c-4178-a2d9-36b32a18e5e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:32.925545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:32.926481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:32.927400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:32.927400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:32.927400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:32.927400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:32.927400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 308/636] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:32.927990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb58bab-18a5-4b0b-87b3-537ce4aeb615 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:32.928561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb58bab-18a5-4b0b-87b3-537ce4aeb615 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:32.928561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cb58bab-18a5-4b0b-87b3-537ce4aeb615 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:32.934212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 309/637] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:32.935333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb58bab-18a5-4b0b-87b3-537ce4aeb615 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:32.998266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.029630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edd7688-304c-4c0c-a34a-264004e98ab7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.038340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cb58bab-18a5-4b0b-87b3-537ce4aeb615 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.063504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae6ad8-6da9-443f-9c46-c05e9498c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.064156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae6ad8-6da9-443f-9c46-c05e9498c331 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.064441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae6ad8-6da9-443f-9c46-c05e9498c331 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.064716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46ae6ad8-6da9-443f-9c46-c05e9498c331 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.065433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae6ad8-6da9-443f-9c46-c05e9498c331 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:33.065765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae6ad8-6da9-443f-9c46-c05e9498c331 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.067138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46ae6ad8-6da9-443f-9c46-c05e9498c331 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.072880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e207a3bd-1a67-490d-9e98-f0ca9d15fef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.073813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207a3bd-1a67-490d-9e98-f0ca9d15fef6 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.074119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207a3bd-1a67-490d-9e98-f0ca9d15fef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.074393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e207a3bd-1a67-490d-9e98-f0ca9d15fef6 None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/9d8387328e4145db80782f2d54c0966d/roles/5b019d81d9cf4a16b6c6d1caafec9081` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.078365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 329/638] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:33.079155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae6ad8-6da9-443f-9c46-c05e9498c331 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.089737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207a3bd-1a67-490d-9e98-f0ca9d15fef6 None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=9d8387328e4145db80782f2d54c0966d, role_id=5b019d81d9cf4a16b6c6d1caafec9081)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.091571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e207a3bd-1a67-490d-9e98-f0ca9d15fef6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.104343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 310/639] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:32 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/9d8387328e4145db80782f2d54c0966d/roles/5b019d81d9cf4a16b6c6d1caafec9081 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:33.106135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d057e06-ea98-46eb-9e9b-baf4afa2d4f4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.106463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d057e06-ea98-46eb-9e9b-baf4afa2d4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.107026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d057e06-ea98-46eb-9e9b-baf4afa2d4f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.125523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6d057e06-ea98-46eb-9e9b-baf4afa2d4f4 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:33.126840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6d057e06-ea98-46eb-9e9b-baf4afa2d4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:33.131802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.199928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46ae6ad8-6da9-443f-9c46-c05e9498c331 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.217535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 311/640] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:33.217535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d057e06-ea98-46eb-9e9b-baf4afa2d4f4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.242012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce71104-bb68-4e20-9765-295bbd73fe28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.242851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce71104-bb68-4e20-9765-295bbd73fe28 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.243237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce71104-bb68-4e20-9765-295bbd73fe28 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.243604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce71104-bb68-4e20-9765-295bbd73fe28 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.244357 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce71104-bb68-4e20-9765-295bbd73fe28 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:33.244679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce71104-bb68-4e20-9765-295bbd73fe28 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.246235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce71104-bb68-4e20-9765-295bbd73fe28 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.262332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 330/641] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:33.263829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce71104-bb68-4e20-9765-295bbd73fe28 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.271715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.349451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d057e06-ea98-46eb-9e9b-baf4afa2d4f4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.385352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce71104-bb68-4e20-9765-295bbd73fe28 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.392182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.396076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8de5b-0775-4d5f-b9ef-3866f5b37cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.397058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8de5b-0775-4d5f-b9ef-3866f5b37cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.397174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8de5b-0775-4d5f-b9ef-3866f5b37cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.397435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06f8de5b-0775-4d5f-b9ef-3866f5b37cc6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.410914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06f8de5b-0775-4d5f-b9ef-3866f5b37cc6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-06f8de5b-0775-4d5f-b9ef-3866f5b37cc6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:33.419249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:33.420284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:33.421137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 312/642] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:33.421874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8de5b-0775-4d5f-b9ef-3866f5b37cc6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.434181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9929cb0-e8da-470a-b0f4-e192e4111157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.435261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9929cb0-e8da-470a-b0f4-e192e4111157 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.435261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9929cb0-e8da-470a-b0f4-e192e4111157 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.435547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9929cb0-e8da-470a-b0f4-e192e4111157 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.447564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9929cb0-e8da-470a-b0f4-e192e4111157 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.449086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9929cb0-e8da-470a-b0f4-e192e4111157 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f9929cb0-e8da-470a-b0f4-e192e4111157 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:33.469233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:33.470570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 331/643] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:33.471515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9929cb0-e8da-470a-b0f4-e192e4111157 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.558007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.563985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06f8de5b-0775-4d5f-b9ef-3866f5b37cc6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.602062 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723fb0f-bd37-4a65-a682-52e3382985f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.603167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723fb0f-bd37-4a65-a682-52e3382985f0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.603918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723fb0f-bd37-4a65-a682-52e3382985f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.603918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1723fb0f-bd37-4a65-a682-52e3382985f0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.604689 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9929cb0-e8da-470a-b0f4-e192e4111157 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.619834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1723fb0f-bd37-4a65-a682-52e3382985f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1723fb0f-bd37-4a65-a682-52e3382985f0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:33.628504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:33.629706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:33.630808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 313/644] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:33.631762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723fb0f-bd37-4a65-a682-52e3382985f0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.653899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7e183-6fbb-4761-956a-64b6fa5f8cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.654699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7e183-6fbb-4761-956a-64b6fa5f8cfc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.656395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7e183-6fbb-4761-956a-64b6fa5f8cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.656395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d7e183-6fbb-4761-956a-64b6fa5f8cfc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.662580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7e183-6fbb-4761-956a-64b6fa5f8cfc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.664037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d7e183-6fbb-4761-956a-64b6fa5f8cfc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-36d7e183-6fbb-4761-956a-64b6fa5f8cfc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:33.677858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:33.679488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:33.680589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:33.680589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:33.680589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:33.680589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:33.680589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:33.680589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:33.680589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:33.680589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:33.680589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 332/645] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:33.680589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7e183-6fbb-4761-956a-64b6fa5f8cfc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.715976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.780641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1723fb0f-bd37-4a65-a682-52e3382985f0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.811212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d7e183-6fbb-4761-956a-64b6fa5f8cfc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.824808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af38c-eaec-4975-b233-302106aa90e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.825655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af38c-eaec-4975-b233-302106aa90e0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.826010 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af38c-eaec-4975-b233-302106aa90e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.826357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9af38c-eaec-4975-b233-302106aa90e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.827195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9af38c-eaec-4975-b233-302106aa90e0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:33.827577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9af38c-eaec-4975-b233-302106aa90e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.829045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9af38c-eaec-4975-b233-302106aa90e0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.842552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 314/646] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:33.842552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af38c-eaec-4975-b233-302106aa90e0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.856754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c2bb7-27c0-4b9e-8de7-bfce9a3216ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:33.857674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c2bb7-27c0-4b9e-8de7-bfce9a3216ff None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:33.858025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c2bb7-27c0-4b9e-8de7-bfce9a3216ff None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:33.858374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512c2bb7-27c0-4b9e-8de7-bfce9a3216ff None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:33.859376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c2bb7-27c0-4b9e-8de7-bfce9a3216ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:33.859764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c2bb7-27c0-4b9e-8de7-bfce9a3216ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:33.861308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512c2bb7-27c0-4b9e-8de7-bfce9a3216ff None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:33.871676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:33.891717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 333/647] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:33.892643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c2bb7-27c0-4b9e-8de7-bfce9a3216ff None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:33.973940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9af38c-eaec-4975-b233-302106aa90e0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:33.995057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.014804 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189112e-670b-443d-a98a-1da212101966 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.015500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189112e-670b-443d-a98a-1da212101966 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.015821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189112e-670b-443d-a98a-1da212101966 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.016055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4189112e-670b-443d-a98a-1da212101966 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.016809 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189112e-670b-443d-a98a-1da212101966 None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:34.017120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189112e-670b-443d-a98a-1da212101966 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.018588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4189112e-670b-443d-a98a-1da212101966 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.020001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512c2bb7-27c0-4b9e-8de7-bfce9a3216ff None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.028262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 315/648] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:34.030716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189112e-670b-443d-a98a-1da212101966 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.063759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427b965-1b64-4b84-8c40-7bd42e545282 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.064036 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427b965-1b64-4b84-8c40-7bd42e545282 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.064287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427b965-1b64-4b84-8c40-7bd42e545282 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.064557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a427b965-1b64-4b84-8c40-7bd42e545282 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.065340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427b965-1b64-4b84-8c40-7bd42e545282 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:34.065809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427b965-1b64-4b84-8c40-7bd42e545282 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.067111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a427b965-1b64-4b84-8c40-7bd42e545282 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.085543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 334/649] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:34.085543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427b965-1b64-4b84-8c40-7bd42e545282 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.138590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.158303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4189112e-670b-443d-a98a-1da212101966 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.196421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1db80-7865-4bb5-abb5-56e42d55e407 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.197415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1db80-7865-4bb5-abb5-56e42d55e407 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.197878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1db80-7865-4bb5-abb5-56e42d55e407 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.198221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c1db80-7865-4bb5-abb5-56e42d55e407 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.211049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1db80-7865-4bb5-abb5-56e42d55e407 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.212913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1c1db80-7865-4bb5-abb5-56e42d55e407 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.231776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a427b965-1b64-4b84-8c40-7bd42e545282 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e1c1db80-7865-4bb5-abb5-56e42d55e407 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:34.232027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:34.233030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:34.234352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:34.234352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:34.234352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:34.234352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:34.238280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 316/650] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:34.239297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1db80-7865-4bb5-abb5-56e42d55e407 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.278683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d5e36-b620-4032-a48a-169ba3fad583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.279650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d5e36-b620-4032-a48a-169ba3fad583 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.280087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d5e36-b620-4032-a48a-169ba3fad583 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.280522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1d5e36-b620-4032-a48a-169ba3fad583 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.295876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.362373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c1db80-7865-4bb5-abb5-56e42d55e407 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.375431 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d5e36-b620-4032-a48a-169ba3fad583 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.377193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d1d5e36-b620-4032-a48a-169ba3fad583 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.408954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6076e-6c54-461d-ae49-40770aa92f2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.409783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6076e-6c54-461d-ae49-40770aa92f2b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.410072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6076e-6c54-461d-ae49-40770aa92f2b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.410350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c6076e-6c54-461d-ae49-40770aa92f2b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.413226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.426053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6076e-6c54-461d-ae49-40770aa92f2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.427802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c6076e-6c54-461d-ae49-40770aa92f2b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-11c6076e-6c54-461d-ae49-40770aa92f2b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:34.449275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:34.450640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:34.452089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:34.452089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:34.452089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:34.452089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:34.452089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 317/651] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:34.452089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6076e-6c54-461d-ae49-40770aa92f2b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.497029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 335/652] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:34.498826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d5e36-b620-4032-a48a-169ba3fad583 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.568614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.607069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c6076e-6c54-461d-ae49-40770aa92f2b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.654478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0662f-3c21-4b3b-bfde-f961ffe709c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.655149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0662f-3c21-4b3b-bfde-f961ffe709c6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.655436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0662f-3c21-4b3b-bfde-f961ffe709c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.655713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39d0662f-3c21-4b3b-bfde-f961ffe709c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.656557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0662f-3c21-4b3b-bfde-f961ffe709c6 None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=3e46ec9c31094d03b6a413e09bc9f699) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:34.656890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0662f-3c21-4b3b-bfde-f961ffe709c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.658355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39d0662f-3c21-4b3b-bfde-f961ffe709c6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.667112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 318/653] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:34 2022] GET /identity/v3/role_assignments?user.id=3e46ec9c31094d03b6a413e09bc9f699&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 608 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:34.668680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d1d5e36-b620-4032-a48a-169ba3fad583 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.669126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0662f-3c21-4b3b-bfde-f961ffe709c6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.735653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5510e7-fc96-4c0a-83a9-cea67bcee640 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.736741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5510e7-fc96-4c0a-83a9-cea67bcee640 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.737213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5510e7-fc96-4c0a-83a9-cea67bcee640 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.737721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5510e7-fc96-4c0a-83a9-cea67bcee640 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.739095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5510e7-fc96-4c0a-83a9-cea67bcee640 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:34.739618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5510e7-fc96-4c0a-83a9-cea67bcee640 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.742250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5510e7-fc96-4c0a-83a9-cea67bcee640 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.763821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 336/654] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:34.765270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5510e7-fc96-4c0a-83a9-cea67bcee640 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.797547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.884035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39d0662f-3c21-4b3b-bfde-f961ffe709c6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.925578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5510e7-fc96-4c0a-83a9-cea67bcee640 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:34.927972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:34.932381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e16fe3-4151-41dc-9864-e15bbf142552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.933068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e16fe3-4151-41dc-9864-e15bbf142552 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.933983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e16fe3-4151-41dc-9864-e15bbf142552 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.934273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e16fe3-4151-41dc-9864-e15bbf142552 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.935025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e16fe3-4151-41dc-9864-e15bbf142552 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:34.935331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e16fe3-4151-41dc-9864-e15bbf142552 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.936742 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e16fe3-4151-41dc-9864-e15bbf142552 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:34.952215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 319/655] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:34.953617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e16fe3-4151-41dc-9864-e15bbf142552 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:34.971563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446902a-485c-497d-a6ff-7562cf6d2d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:34.972478 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446902a-485c-497d-a6ff-7562cf6d2d35 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:34.972761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446902a-485c-497d-a6ff-7562cf6d2d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:34.973035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3446902a-485c-497d-a6ff-7562cf6d2d35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:34.986553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446902a-485c-497d-a6ff-7562cf6d2d35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:34.988265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3446902a-485c-497d-a6ff-7562cf6d2d35 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-3446902a-485c-497d-a6ff-7562cf6d2d35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:35.008592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:35.010384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:35.012425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:35.012425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:35.012425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:35.012425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:35.012425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 337/656] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:35.012425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446902a-485c-497d-a6ff-7562cf6d2d35 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.073747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e16fe3-4151-41dc-9864-e15bbf142552 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:35.089150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:35.115885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8cf9f-cb30-47de-9dee-bce424bdc783 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:35.116932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8cf9f-cb30-47de-9dee-bce424bdc783 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.117401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8cf9f-cb30-47de-9dee-bce424bdc783 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.117704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd8cf9f-cb30-47de-9dee-bce424bdc783 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.126013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8cf9f-cb30-47de-9dee-bce424bdc783 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:35.127478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd8cf9f-cb30-47de-9dee-bce424bdc783 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:35.141306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3446902a-485c-497d-a6ff-7562cf6d2d35 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-5dd8cf9f-cb30-47de-9dee-bce424bdc783 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:35.142453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:35.143386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:35.144327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 320/657] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:35.144920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8cf9f-cb30-47de-9dee-bce424bdc783 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.191215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b473d5c-cab1-439b-a21f-82608cdafcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:35.193126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b473d5c-cab1-439b-a21f-82608cdafcfa None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.193625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b473d5c-cab1-439b-a21f-82608cdafcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.194417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b473d5c-cab1-439b-a21f-82608cdafcfa None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.206566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b473d5c-cab1-439b-a21f-82608cdafcfa None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:35.209221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b473d5c-cab1-439b-a21f-82608cdafcfa None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-9b473d5c-cab1-439b-a21f-82608cdafcfa None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:35.228904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:35.229965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:35.231683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:35.231683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:35.231683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:35.231683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:35.231683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:35.231683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:35.231683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:35.231683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:35.231683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 20 19:10:35.231683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:35.235386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 338/658] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/users/neutron => generated 85 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:35.236197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b473d5c-cab1-439b-a21f-82608cdafcfa None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.253816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:35.255506 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd8cf9f-cb30-47de-9dee-bce424bdc783 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:35.291901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4227b43b-c297-4128-adf6-87c4906d755a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:35.292572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b43b-c297-4128-adf6-87c4906d755a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.293441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b43b-c297-4128-adf6-87c4906d755a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.293753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4227b43b-c297-4128-adf6-87c4906d755a None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.294487 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227b43b-c297-4128-adf6-87c4906d755a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:35.294794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227b43b-c297-4128-adf6-87c4906d755a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:35.296185 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4227b43b-c297-4128-adf6-87c4906d755a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:35.312994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 321/659] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:35.312994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066504a9-806d-408d-883f-94ee77ac94d3 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.312994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066504a9-806d-408d-883f-94ee77ac94d3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.312994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066504a9-806d-408d-883f-94ee77ac94d3 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.313405 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 322/660] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:35.314160 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066504a9-806d-408d-883f-94ee77ac94d3 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.353674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:35.364576 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b473d5c-cab1-439b-a21f-82608cdafcfa None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:35.399050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049741ae-f03d-4db0-9b20-773ddd2309b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:35.399723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049741ae-f03d-4db0-9b20-773ddd2309b0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.400002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049741ae-f03d-4db0-9b20-773ddd2309b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.400275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-049741ae-f03d-4db0-9b20-773ddd2309b0 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.401103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049741ae-f03d-4db0-9b20-773ddd2309b0 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:35.401407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049741ae-f03d-4db0-9b20-773ddd2309b0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:35.403499 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-049741ae-f03d-4db0-9b20-773ddd2309b0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:35.426555 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 339/661] 10.222.0.69 () {56 vars in 1137 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:35.427749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049741ae-f03d-4db0-9b20-773ddd2309b0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.439624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066504a9-806d-408d-883f-94ee77ac94d3 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:35.478119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb41a0f5-17dd-405c-b988-aa660e81308a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:35.478782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41a0f5-17dd-405c-b988-aa660e81308a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.479063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41a0f5-17dd-405c-b988-aa660e81308a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.479336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb41a0f5-17dd-405c-b988-aa660e81308a None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.480475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41a0f5-17dd-405c-b988-aa660e81308a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:35.480879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41a0f5-17dd-405c-b988-aa660e81308a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:35.482628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb41a0f5-17dd-405c-b988-aa660e81308a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:35.507142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 323/662] 10.222.0.69 () {56 vars in 1139 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:35.508354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841815c7-36e3-4521-a0cd-2de3126fbc37 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.508495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:35.508638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841815c7-36e3-4521-a0cd-2de3126fbc37 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.508912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841815c7-36e3-4521-a0cd-2de3126fbc37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.527891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-841815c7-36e3-4521-a0cd-2de3126fbc37 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:35.529075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-841815c7-36e3-4521-a0cd-2de3126fbc37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:35.537409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-049741ae-f03d-4db0-9b20-773ddd2309b0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:35.573921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f18d3-4055-497c-b160-7df42e81f100 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:35.575550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f18d3-4055-497c-b160-7df42e81f100 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.575877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f18d3-4055-497c-b160-7df42e81f100 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.576129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6f18d3-4055-497c-b160-7df42e81f100 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.584744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f18d3-4055-497c-b160-7df42e81f100 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:35.586239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6f18d3-4055-497c-b160-7df42e81f100 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6d6f18d3-4055-497c-b160-7df42e81f100 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:35.601908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:35.603019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:35.603929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:35.603929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:35.603929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:35.603929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:35.603929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:35.603929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:35.603929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:35.603929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:35.603929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 340/663] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:35.603929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f18d3-4055-497c-b160-7df42e81f100 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.611859 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 324/664] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:35.612713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841815c7-36e3-4521-a0cd-2de3126fbc37 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.642003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:35.713384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6f18d3-4055-497c-b160-7df42e81f100 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:35.733609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841815c7-36e3-4521-a0cd-2de3126fbc37 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:35.749471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:35.755754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14319420-3ba6-4c97-9456-9853787ed38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:35.756489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14319420-3ba6-4c97-9456-9853787ed38f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.756841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14319420-3ba6-4c97-9456-9853787ed38f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.757153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14319420-3ba6-4c97-9456-9853787ed38f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.772031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b57f1-5fb4-4019-9d31-6a967b291822 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:35.772837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b57f1-5fb4-4019-9d31-6a967b291822 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:35.773181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b57f1-5fb4-4019-9d31-6a967b291822 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:35.773519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678b57f1-5fb4-4019-9d31-6a967b291822 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:35.837592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14319420-3ba6-4c97-9456-9853787ed38f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:35.839041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14319420-3ba6-4c97-9456-9853787ed38f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:35.852186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b57f1-5fb4-4019-9d31-6a967b291822 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:35.853682 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678b57f1-5fb4-4019-9d31-6a967b291822 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:35.876619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:35.951007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 341/665] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:35.951853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14319420-3ba6-4c97-9456-9853787ed38f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.960110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 325/666] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:35.961218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b57f1-5fb4-4019-9d31-6a967b291822 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:35.989199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.071534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678b57f1-5fb4-4019-9d31-6a967b291822 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.072197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14319420-3ba6-4c97-9456-9853787ed38f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.093769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.109217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575020d1-e518-4d0e-9369-0537b1395f7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.109546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845ce78-c9f2-41e7-b397-0bec2873ce70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.109908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575020d1-e518-4d0e-9369-0537b1395f7e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.110192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575020d1-e518-4d0e-9369-0537b1395f7e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.110475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575020d1-e518-4d0e-9369-0537b1395f7e None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.110708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845ce78-c9f2-41e7-b397-0bec2873ce70 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.111081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845ce78-c9f2-41e7-b397-0bec2873ce70 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.111255 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575020d1-e518-4d0e-9369-0537b1395f7e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:36.111568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575020d1-e518-4d0e-9369-0537b1395f7e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.111687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8845ce78-c9f2-41e7-b397-0bec2873ce70 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.113005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575020d1-e518-4d0e-9369-0537b1395f7e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.122600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8845ce78-c9f2-41e7-b397-0bec2873ce70 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:36.131320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 342/667] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:36.131320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575020d1-e518-4d0e-9369-0537b1395f7e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8845ce78-c9f2-41e7-b397-0bec2873ce70 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:36.131571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:36.132600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:36.133541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:36.136938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 326/668] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:35 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:36.137954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845ce78-c9f2-41e7-b397-0bec2873ce70 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.239982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.250648 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8845ce78-c9f2-41e7-b397-0bec2873ce70 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.254445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575020d1-e518-4d0e-9369-0537b1395f7e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.289348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535e420-119e-40d7-b32c-4d6508c2b364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.290216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535e420-119e-40d7-b32c-4d6508c2b364 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.290453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535e420-119e-40d7-b32c-4d6508c2b364 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.290727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6535e420-119e-40d7-b32c-4d6508c2b364 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.296752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e710e-8577-4f96-9021-2053917652f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.297603 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e710e-8577-4f96-9021-2053917652f9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.297870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e710e-8577-4f96-9021-2053917652f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.298094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a9e710e-8577-4f96-9021-2053917652f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.302679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535e420-119e-40d7-b32c-4d6508c2b364 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.304149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6535e420-119e-40d7-b32c-4d6508c2b364 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.309851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e710e-8577-4f96-9021-2053917652f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.311273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a9e710e-8577-4f96-9021-2053917652f9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6535e420-119e-40d7-b32c-4d6508c2b364 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:36.329027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:36.330069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:36.335953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:36.335953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:36.335953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:36.335953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:36.335953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 327/669] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-5a9e710e-8577-4f96-9021-2053917652f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:36.336227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:36.337351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:36.338279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:36.338279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:36.338279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:36.338279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:36.339253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535e420-119e-40d7-b32c-4d6508c2b364 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.339392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 343/670] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:36.339392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e710e-8577-4f96-9021-2053917652f9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.372133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.451775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6535e420-119e-40d7-b32c-4d6508c2b364 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.452461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a9e710e-8577-4f96-9021-2053917652f9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.488834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150b6a7-f0bf-4747-bd3b-e9f509f50263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.489499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150b6a7-f0bf-4747-bd3b-e9f509f50263 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.489811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150b6a7-f0bf-4747-bd3b-e9f509f50263 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.490139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0150b6a7-f0bf-4747-bd3b-e9f509f50263 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.490916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150b6a7-f0bf-4747-bd3b-e9f509f50263 None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=36955d7d4505457696ab6177adb999a9) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:36.491226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150b6a7-f0bf-4747-bd3b-e9f509f50263 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.491774 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb2975-158e-4a19-a8fd-350f7cfb7240 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.492436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb2975-158e-4a19-a8fd-350f7cfb7240 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.492665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0150b6a7-f0bf-4747-bd3b-e9f509f50263 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.492760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb2975-158e-4a19-a8fd-350f7cfb7240 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.492975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2eb2975-158e-4a19-a8fd-350f7cfb7240 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.493736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb2975-158e-4a19-a8fd-350f7cfb7240 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:36.494043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb2975-158e-4a19-a8fd-350f7cfb7240 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.495379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2eb2975-158e-4a19-a8fd-350f7cfb7240 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.510544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 328/671] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/role_assignments?user.id=36955d7d4505457696ab6177adb999a9&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 608 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:36.510544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150b6a7-f0bf-4747-bd3b-e9f509f50263 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.515285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 344/672] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:36.516347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb2975-158e-4a19-a8fd-350f7cfb7240 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.527052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.625951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.634555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0150b6a7-f0bf-4747-bd3b-e9f509f50263 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.663106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2eb2975-158e-4a19-a8fd-350f7cfb7240 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.682535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68707837-07df-47f1-add9-b52aad9b9093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.684569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68707837-07df-47f1-add9-b52aad9b9093 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.684923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68707837-07df-47f1-add9-b52aad9b9093 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.686867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68707837-07df-47f1-add9-b52aad9b9093 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.686867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68707837-07df-47f1-add9-b52aad9b9093 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:36.686867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68707837-07df-47f1-add9-b52aad9b9093 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.688577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68707837-07df-47f1-add9-b52aad9b9093 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.705881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 329/673] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:36.709924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801e255-0289-45d5-be29-4e4762a5e5fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.710591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801e255-0289-45d5-be29-4e4762a5e5fe None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.710872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801e255-0289-45d5-be29-4e4762a5e5fe None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.711147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801e255-0289-45d5-be29-4e4762a5e5fe None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.711273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68707837-07df-47f1-add9-b52aad9b9093 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.711914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801e255-0289-45d5-be29-4e4762a5e5fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:36.712167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801e255-0289-45d5-be29-4e4762a5e5fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.713529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801e255-0289-45d5-be29-4e4762a5e5fe None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.727741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 345/674] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:36.728571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801e255-0289-45d5-be29-4e4762a5e5fe None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.819846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:36.842013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68707837-07df-47f1-add9-b52aad9b9093 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.861982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801e255-0289-45d5-be29-4e4762a5e5fe None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:36.889354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72c63d-13cc-4b6b-9c5a-f33c7b417d8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.891100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72c63d-13cc-4b6b-9c5a-f33c7b417d8a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.891755 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72c63d-13cc-4b6b-9c5a-f33c7b417d8a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.892626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b72c63d-13cc-4b6b-9c5a-f33c7b417d8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.900304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c200ff4-7ca6-44a7-9d3d-acf87f6dfb77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:36.901188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c200ff4-7ca6-44a7-9d3d-acf87f6dfb77 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:36.901880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c200ff4-7ca6-44a7-9d3d-acf87f6dfb77 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:36.901992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c200ff4-7ca6-44a7-9d3d-acf87f6dfb77 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:36.906250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72c63d-13cc-4b6b-9c5a-f33c7b417d8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.908406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b72c63d-13cc-4b6b-9c5a-f33c7b417d8a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.911694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c200ff4-7ca6-44a7-9d3d-acf87f6dfb77 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:36.913196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c200ff4-7ca6-44a7-9d3d-acf87f6dfb77 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-9c200ff4-7ca6-44a7-9d3d-acf87f6dfb77 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:36.928516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:36.929463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:36.930648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:36.930648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:36.930648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:36.930648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:36.930648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:36.930648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:36.930648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:36.930648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:36.930648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 346/675] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1b72c63d-13cc-4b6b-9c5a-f33c7b417d8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:36.931756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:36.932603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:36.934106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:36.934106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:36.934106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:36.934106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:36.937649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c200ff4-7ca6-44a7-9d3d-acf87f6dfb77 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.940652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 330/676] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:36.943444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72c63d-13cc-4b6b-9c5a-f33c7b417d8a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:36.957777 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.058518 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c200ff4-7ca6-44a7-9d3d-acf87f6dfb77 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.064502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b72c63d-13cc-4b6b-9c5a-f33c7b417d8a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.085003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.096345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b801f62-d371-4126-931f-759637baa6b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.097324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b801f62-d371-4126-931f-759637baa6b1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.097708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b801f62-d371-4126-931f-759637baa6b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.098052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b801f62-d371-4126-931f-759637baa6b1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.100134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30a0c8-d7e5-4abd-9597-63fad5c913ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.100979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30a0c8-d7e5-4abd-9597-63fad5c913ea None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.101256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30a0c8-d7e5-4abd-9597-63fad5c913ea None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.101557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e30a0c8-d7e5-4abd-9597-63fad5c913ea None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.102350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30a0c8-d7e5-4abd-9597-63fad5c913ea None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:37.102661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30a0c8-d7e5-4abd-9597-63fad5c913ea None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.104115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e30a0c8-d7e5-4abd-9597-63fad5c913ea None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.108369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b801f62-d371-4126-931f-759637baa6b1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.109050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b801f62-d371-4126-931f-759637baa6b1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.123179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 331/677] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:37.123179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30a0c8-d7e5-4abd-9597-63fad5c913ea None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6b801f62-d371-4126-931f-759637baa6b1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:37.124498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:37.125565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:37.126607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:37.126607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:37.126607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:37.126607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:37.126607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:37.126607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:37.126607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:37.126607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:37.126607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 347/678] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:37.127207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca536a79-9750-49ab-80e6-0677bf26de20 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.127324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca536a79-9750-49ab-80e6-0677bf26de20 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.127609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca536a79-9750-49ab-80e6-0677bf26de20 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.132977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 348/679] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:37 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:37.133860 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca536a79-9750-49ab-80e6-0677bf26de20 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.195254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.231475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e30a0c8-d7e5-4abd-9597-63fad5c913ea None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.249177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca536a79-9750-49ab-80e6-0677bf26de20 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.265001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a4c2-f310-4348-8c7a-a406bf11144c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.265787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a4c2-f310-4348-8c7a-a406bf11144c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.266130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a4c2-f310-4348-8c7a-a406bf11144c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.266469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec17a4c2-f310-4348-8c7a-a406bf11144c None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.267251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17a4c2-f310-4348-8c7a-a406bf11144c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:37.267621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17a4c2-f310-4348-8c7a-a406bf11144c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.269065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec17a4c2-f310-4348-8c7a-a406bf11144c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.282161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 332/680] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:37.282920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a4c2-f310-4348-8c7a-a406bf11144c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.290303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fb051a-0140-4769-961f-1f5b32125632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.291108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb051a-0140-4769-961f-1f5b32125632 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.291502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb051a-0140-4769-961f-1f5b32125632 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.291864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fb051a-0140-4769-961f-1f5b32125632 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.292839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb051a-0140-4769-961f-1f5b32125632 None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=3a1a88089d384964bd9774eda4c5d8a9) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:37.293936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb051a-0140-4769-961f-1f5b32125632 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.295732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fb051a-0140-4769-961f-1f5b32125632 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.309551 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 349/681] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/role_assignments?user.id=3a1a88089d384964bd9774eda4c5d8a9&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 608 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:37.309551 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f62d4-50bb-46ce-8dcb-642b613c3110 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.309551 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f62d4-50bb-46ce-8dcb-642b613c3110 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.309551 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784f62d4-50bb-46ce-8dcb-642b613c3110 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.333381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-784f62d4-50bb-46ce-8dcb-642b613c3110 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:37.335106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-784f62d4-50bb-46ce-8dcb-642b613c3110 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:37.397904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.418455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 350/682] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:37.419723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f62d4-50bb-46ce-8dcb-642b613c3110 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.428756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec17a4c2-f310-4348-8c7a-a406bf11144c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.464959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.476053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a078e2-c68d-4f94-bad4-14d7ee53460d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.477143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a078e2-c68d-4f94-bad4-14d7ee53460d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.477622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a078e2-c68d-4f94-bad4-14d7ee53460d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.478048 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a078e2-c68d-4f94-bad4-14d7ee53460d None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.479679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a078e2-c68d-4f94-bad4-14d7ee53460d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:37.480936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a078e2-c68d-4f94-bad4-14d7ee53460d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.483291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a078e2-c68d-4f94-bad4-14d7ee53460d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.500255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 333/683] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:37.502239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a078e2-c68d-4f94-bad4-14d7ee53460d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.552277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784f62d4-50bb-46ce-8dcb-642b613c3110 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.596176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aa8cc-bad6-4505-910a-e345cd033934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.596374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aa8cc-bad6-4505-910a-e345cd033934 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.596629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aa8cc-bad6-4505-910a-e345cd033934 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.596896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004aa8cc-bad6-4505-910a-e345cd033934 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.605673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aa8cc-bad6-4505-910a-e345cd033934 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.607103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004aa8cc-bad6-4505-910a-e345cd033934 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-004aa8cc-bad6-4505-910a-e345cd033934 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:37.622773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:37.623859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 351/684] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:37.625391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aa8cc-bad6-4505-910a-e345cd033934 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.639415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a078e2-c68d-4f94-bad4-14d7ee53460d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.675582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.691051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dda86-af53-4290-9b4a-5b3c62385650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.691995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dda86-af53-4290-9b4a-5b3c62385650 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.692397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dda86-af53-4290-9b4a-5b3c62385650 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.692675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891dda86-af53-4290-9b4a-5b3c62385650 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.711211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dda86-af53-4290-9b4a-5b3c62385650 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.712734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-891dda86-af53-4290-9b4a-5b3c62385650 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-891dda86-af53-4290-9b4a-5b3c62385650 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:37.737715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:37.738870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:37.739985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:37.739985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:37.739985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:37.739985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:37.740341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 334/685] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 239 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:37.741170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dda86-af53-4290-9b4a-5b3c62385650 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.755789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004aa8cc-bad6-4505-910a-e345cd033934 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.790628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.794598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf93ac1-62df-4f6f-a9c6-0019bbe478c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.795381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf93ac1-62df-4f6f-a9c6-0019bbe478c6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.795771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf93ac1-62df-4f6f-a9c6-0019bbe478c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.796168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf93ac1-62df-4f6f-a9c6-0019bbe478c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.797121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf93ac1-62df-4f6f-a9c6-0019bbe478c6 None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=c94d9030f61a4e1f8b3d75ef8925bdd0) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:37.797563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf93ac1-62df-4f6f-a9c6-0019bbe478c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.799105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbf93ac1-62df-4f6f-a9c6-0019bbe478c6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.805686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 352/686] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/role_assignments?user.id=c94d9030f61a4e1f8b3d75ef8925bdd0&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:37.806523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf93ac1-62df-4f6f-a9c6-0019bbe478c6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.860910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891dda86-af53-4290-9b4a-5b3c62385650 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.918149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf93ac1-62df-4f6f-a9c6-0019bbe478c6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:37.921830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:37.924717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd971b62-c701-4cfc-8744-019fb5271dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.925426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd971b62-c701-4cfc-8744-019fb5271dbb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.925798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd971b62-c701-4cfc-8744-019fb5271dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.926073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd971b62-c701-4cfc-8744-019fb5271dbb None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.926924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd971b62-c701-4cfc-8744-019fb5271dbb None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:37.927238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd971b62-c701-4cfc-8744-019fb5271dbb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.928687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd971b62-c701-4cfc-8744-019fb5271dbb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.960241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61c8d6-468f-4668-87e4-81ef86c5d2ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:37.961096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61c8d6-468f-4668-87e4-81ef86c5d2ae None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:37.961484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61c8d6-468f-4668-87e4-81ef86c5d2ae None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:37.961914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e61c8d6-468f-4668-87e4-81ef86c5d2ae None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:37.962780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61c8d6-468f-4668-87e4-81ef86c5d2ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:37.963164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61c8d6-468f-4668-87e4-81ef86c5d2ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:37.964843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e61c8d6-468f-4668-87e4-81ef86c5d2ae None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:37.966868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 335/687] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:37.972124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd971b62-c701-4cfc-8744-019fb5271dbb None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:37.979658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 353/688] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:37.988685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61c8d6-468f-4668-87e4-81ef86c5d2ae None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.125015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e61c8d6-468f-4668-87e4-81ef86c5d2ae None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.127357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:38.131989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd971b62-c701-4cfc-8744-019fb5271dbb None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.165633 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07d51e-f0e4-4496-b8b5-ca4a2f103435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.166651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d51e-f0e4-4496-b8b5-ca4a2f103435 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.167067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d51e-f0e4-4496-b8b5-ca4a2f103435 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.167495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb07d51e-f0e4-4496-b8b5-ca4a2f103435 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.169553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07d51e-f0e4-4496-b8b5-ca4a2f103435 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.171257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb07d51e-f0e4-4496-b8b5-ca4a2f103435 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:38.171948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123c48a-036f-4c80-b6d6-22ee1cf9b0d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.173187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123c48a-036f-4c80-b6d6-22ee1cf9b0d7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.173605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123c48a-036f-4c80-b6d6-22ee1cf9b0d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.173984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c123c48a-036f-4c80-b6d6-22ee1cf9b0d7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.200246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 354/689] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:10:37 2022] POST /identity/v3/users => generated 254 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:38.280333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123c48a-036f-4c80-b6d6-22ee1cf9b0d7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.281846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c123c48a-036f-4c80-b6d6-22ee1cf9b0d7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:38.391879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:38.398832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 336/690] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:10:38.409059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07d51e-f0e4-4496-b8b5-ca4a2f103435 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.444827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:38.540494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb07d51e-f0e4-4496-b8b5-ca4a2f103435 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.577193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaba0a2-3040-4561-af12-8f5b8f958b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.578015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaba0a2-3040-4561-af12-8f5b8f958b2c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.578294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaba0a2-3040-4561-af12-8f5b8f958b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.578565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaba0a2-3040-4561-af12-8f5b8f958b2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.590388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaba0a2-3040-4561-af12-8f5b8f958b2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.591862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaba0a2-3040-4561-af12-8f5b8f958b2c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-eaaba0a2-3040-4561-af12-8f5b8f958b2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:38.610426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:38.611388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:38.612303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:38.612303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:38.612303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:38.612303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:38.612303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 355/691] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:38.617435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123c48a-036f-4c80-b6d6-22ee1cf9b0d7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.722159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c123c48a-036f-4c80-b6d6-22ee1cf9b0d7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.760599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c405986-d391-4a5a-b6d0-b64992201972 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.761268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c405986-d391-4a5a-b6d0-b64992201972 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.761646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c405986-d391-4a5a-b6d0-b64992201972 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.761902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c405986-d391-4a5a-b6d0-b64992201972 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.762614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c405986-d391-4a5a-b6d0-b64992201972 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:38.762919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c405986-d391-4a5a-b6d0-b64992201972 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.764300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c405986-d391-4a5a-b6d0-b64992201972 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:38.776684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 337/692] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:38.782802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaba0a2-3040-4561-af12-8f5b8f958b2c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:38.867128 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:38.906378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaba0a2-3040-4561-af12-8f5b8f958b2c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:38.923496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49fb38-234a-4fec-95b0-8056fc49a0e3 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.923804 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49fb38-234a-4fec-95b0-8056fc49a0e3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.924151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c49fb38-234a-4fec-95b0-8056fc49a0e3 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.925769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 338/693] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:38.930716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3f659-3277-47f8-9db8-1c5e16298edf None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.931004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3f659-3277-47f8-9db8-1c5e16298edf None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.931280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc3f659-3277-47f8-9db8-1c5e16298edf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.961464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fdc3f659-3277-47f8-9db8-1c5e16298edf None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:38.962680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fdc3f659-3277-47f8-9db8-1c5e16298edf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:38.964562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49ee10-7985-4968-8eaf-985f313c0e10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:38.965975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49ee10-7985-4968-8eaf-985f313c0e10 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:38.966251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49ee10-7985-4968-8eaf-985f313c0e10 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:38.966524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d49ee10-7985-4968-8eaf-985f313c0e10 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:38.978744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49ee10-7985-4968-8eaf-985f313c0e10 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:38.981168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d49ee10-7985-4968-8eaf-985f313c0e10 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-5d49ee10-7985-4968-8eaf-985f313c0e10 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:39.006230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:39.007375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:39.008510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:39.008510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:39.008510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:39.008510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:39.008510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:39.008510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:39.008510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:39.008510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:39.010961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 356/694] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:39.018109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49ee10-7985-4968-8eaf-985f313c0e10 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.051273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:39.054244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 339/695] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:39.108139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3f659-3277-47f8-9db8-1c5e16298edf None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.149007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d49ee10-7985-4968-8eaf-985f313c0e10 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.185797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:39.192814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc754164-3102-4fe8-af49-28a49977fa5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.193486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc754164-3102-4fe8-af49-28a49977fa5f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.193790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc754164-3102-4fe8-af49-28a49977fa5f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.194064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc754164-3102-4fe8-af49-28a49977fa5f None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.195792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc754164-3102-4fe8-af49-28a49977fa5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:39.196105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc754164-3102-4fe8-af49-28a49977fa5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:39.197555 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc754164-3102-4fe8-af49-28a49977fa5f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:39.211929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 357/696] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:39.218391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc754164-3102-4fe8-af49-28a49977fa5f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.341046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc754164-3102-4fe8-af49-28a49977fa5f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.380693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c9759-6f07-4a1b-9251-75c66c454e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.381769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c9759-6f07-4a1b-9251-75c66c454e01 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.382524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c9759-6f07-4a1b-9251-75c66c454e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.383027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07c9759-6f07-4a1b-9251-75c66c454e01 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.384047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c9759-6f07-4a1b-9251-75c66c454e01 None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=9d8387328e4145db80782f2d54c0966d) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:39.384336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc3f659-3277-47f8-9db8-1c5e16298edf None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.384800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c9759-6f07-4a1b-9251-75c66c454e01 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:39.387326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07c9759-6f07-4a1b-9251-75c66c454e01 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:39.395240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 358/697] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/role_assignments?user.id=9d8387328e4145db80782f2d54c0966d&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 608 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:39.465155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e753e9-a622-4e6e-8fda-e76c6d373d3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.466562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e753e9-a622-4e6e-8fda-e76c6d373d3d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.466864 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e753e9-a622-4e6e-8fda-e76c6d373d3d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.467146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e753e9-a622-4e6e-8fda-e76c6d373d3d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.490594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e753e9-a622-4e6e-8fda-e76c6d373d3d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-66e753e9-a622-4e6e-8fda-e76c6d373d3d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:39.498677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:39.499717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:39.500551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 340/698] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:39.507551 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c9759-6f07-4a1b-9251-75c66c454e01 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.581321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:39.643708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07c9759-6f07-4a1b-9251-75c66c454e01 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.687331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a0ba92-516e-4bf8-8ee3-5d3011008699 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.688021 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0ba92-516e-4bf8-8ee3-5d3011008699 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.688308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0ba92-516e-4bf8-8ee3-5d3011008699 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.688586 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a0ba92-516e-4bf8-8ee3-5d3011008699 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.689345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0ba92-516e-4bf8-8ee3-5d3011008699 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:39.689689 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0ba92-516e-4bf8-8ee3-5d3011008699 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:39.691064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a0ba92-516e-4bf8-8ee3-5d3011008699 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:39.701019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 359/699] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:39.706477 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e753e9-a622-4e6e-8fda-e76c6d373d3d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.766045 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e454cc7-8604-4ce6-88a3-5a1b17842ce0 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.766426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e454cc7-8604-4ce6-88a3-5a1b17842ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.766710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e454cc7-8604-4ce6-88a3-5a1b17842ce0 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.768356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 360/700] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:39.773592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1d1b9-78f7-4f33-aaef-d25d65e11b91 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.773882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1d1b9-78f7-4f33-aaef-d25d65e11b91 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.774166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8d1d1b9-78f7-4f33-aaef-d25d65e11b91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.779929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:39.796327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a8d1d1b9-78f7-4f33-aaef-d25d65e11b91 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:39.797545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a8d1d1b9-78f7-4f33-aaef-d25d65e11b91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:39.845095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e753e9-a622-4e6e-8fda-e76c6d373d3d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:39.875035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 361/701] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 20 19:10:39.899872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aee786-bb14-4bf5-a5db-aa38cd96e5d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:39.900984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aee786-bb14-4bf5-a5db-aa38cd96e5d7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:39.901413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aee786-bb14-4bf5-a5db-aa38cd96e5d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:39.901858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99aee786-bb14-4bf5-a5db-aa38cd96e5d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:39.909594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1d1b9-78f7-4f33-aaef-d25d65e11b91 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:39.924356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aee786-bb14-4bf5-a5db-aa38cd96e5d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:39.926118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99aee786-bb14-4bf5-a5db-aa38cd96e5d7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-99aee786-bb14-4bf5-a5db-aa38cd96e5d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:39.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:39.951545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:39.952638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:39.952638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:39.952638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:39.952638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:39.954817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 341/702] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:39.960581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aee786-bb14-4bf5-a5db-aa38cd96e5d7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.053742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.075836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99aee786-bb14-4bf5-a5db-aa38cd96e5d7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.085948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.125926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8d1d1b9-78f7-4f33-aaef-d25d65e11b91 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.132631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79336ec6-4857-47fa-8484-5893f3f62a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.133476 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79336ec6-4857-47fa-8484-5893f3f62a8d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.133795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79336ec6-4857-47fa-8484-5893f3f62a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.134077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79336ec6-4857-47fa-8484-5893f3f62a8d None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.134815 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79336ec6-4857-47fa-8484-5893f3f62a8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:40.135128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79336ec6-4857-47fa-8484-5893f3f62a8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.136700 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79336ec6-4857-47fa-8484-5893f3f62a8d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.163352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 342/703] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:40.168561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79336ec6-4857-47fa-8484-5893f3f62a8d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.177538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d7ceb-7f5a-4e5c-a922-64b1d88536c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.177538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d7ceb-7f5a-4e5c-a922-64b1d88536c9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.177538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d7ceb-7f5a-4e5c-a922-64b1d88536c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.177538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631d7ceb-7f5a-4e5c-a922-64b1d88536c9 None admin] PATH_INFO: `/v3/services/network` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.177926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d7ceb-7f5a-4e5c-a922-64b1d88536c9 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.180097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631d7ceb-7f5a-4e5c-a922-64b1d88536c9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-631d7ceb-7f5a-4e5c-a922-64b1d88536c9 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:40.200940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:40.202094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:10:40.202924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:40.211180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 362/704] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:39 2022] GET /identity/v3/services/network => generated 88 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:40.213411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d7ceb-7f5a-4e5c-a922-64b1d88536c9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.377487 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79336ec6-4857-47fa-8484-5893f3f62a8d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.420044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631d7ceb-7f5a-4e5c-a922-64b1d88536c9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.444951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.445821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb22b64-eab6-4305-ad98-12da83183f89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.446630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb22b64-eab6-4305-ad98-12da83183f89 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.446915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb22b64-eab6-4305-ad98-12da83183f89 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.447193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb22b64-eab6-4305-ad98-12da83183f89 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.463871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb22b64-eab6-4305-ad98-12da83183f89 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.465657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb22b64-eab6-4305-ad98-12da83183f89 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-4bb22b64-eab6-4305-ad98-12da83183f89 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:40.490617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:40.491636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:40.492568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 343/705] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/users/nova => generated 82 bytes in 324 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:40.497531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98daa72-0fa1-4267-b4ff-f35b80e04128 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.497531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98daa72-0fa1-4267-b4ff-f35b80e04128 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.497531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98daa72-0fa1-4267-b4ff-f35b80e04128 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.497531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98daa72-0fa1-4267-b4ff-f35b80e04128 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.498180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98daa72-0fa1-4267-b4ff-f35b80e04128 None admin] RBAC: Adding query filter params (name=network) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:40.498180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98daa72-0fa1-4267-b4ff-f35b80e04128 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.498484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98daa72-0fa1-4267-b4ff-f35b80e04128 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.500289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb22b64-eab6-4305-ad98-12da83183f89 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.515868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 363/706] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/services?name=network => generated 124 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:40.522408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98daa72-0fa1-4267-b4ff-f35b80e04128 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.540431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.697843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb22b64-eab6-4305-ad98-12da83183f89 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.719387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98daa72-0fa1-4267-b4ff-f35b80e04128 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:40.737417 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdae41cd-d9b7-473a-90e7-a9b0a867901e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.738191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae41cd-d9b7-473a-90e7-a9b0a867901e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.738480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae41cd-d9b7-473a-90e7-a9b0a867901e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.738758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdae41cd-d9b7-473a-90e7-a9b0a867901e None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.739733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae41cd-d9b7-473a-90e7-a9b0a867901e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:40.740045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae41cd-d9b7-473a-90e7-a9b0a867901e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.741652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdae41cd-d9b7-473a-90e7-a9b0a867901e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.764083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 344/707] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:40.765579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007835-63b7-46c6-908d-8657c6dfe382 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.765880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007835-63b7-46c6-908d-8657c6dfe382 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.766157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3007835-63b7-46c6-908d-8657c6dfe382 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.767704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 345/708] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:40.770218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3007835-63b7-46c6-908d-8657c6dfe382 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:40.785255 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63df414-33f7-4a52-9c46-c3f65fdd1440 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:40.786664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63df414-33f7-4a52-9c46-c3f65fdd1440 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.787023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63df414-33f7-4a52-9c46-c3f65fdd1440 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.787375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63df414-33f7-4a52-9c46-c3f65fdd1440 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.788163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63df414-33f7-4a52-9c46-c3f65fdd1440 None admin] RBAC: Adding query filter params (type=network) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:40.788542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63df414-33f7-4a52-9c46-c3f65fdd1440 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:40.790745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63df414-33f7-4a52-9c46-c3f65fdd1440 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:40.798895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 364/709] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/services?type=network => generated 124 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:40.800739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd4277-4479-43f2-b8ee-c1274e3a634d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:40.801079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd4277-4479-43f2-b8ee-c1274e3a634d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:40.801369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd4277-4479-43f2-b8ee-c1274e3a634d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:40.838871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ebfd4277-4479-43f2-b8ee-c1274e3a634d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:40.841319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ebfd4277-4479-43f2-b8ee-c1274e3a634d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:40.942181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.944660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:40.976189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3007835-63b7-46c6-908d-8657c6dfe382 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.004102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 365/710] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:41.018080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb4f637-da88-4a5e-8c1f-c4d4cef87e68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.019142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4f637-da88-4a5e-8c1f-c4d4cef87e68 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.019620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4f637-da88-4a5e-8c1f-c4d4cef87e68 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.020203 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb4f637-da88-4a5e-8c1f-c4d4cef87e68 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.031373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd4277-4479-43f2-b8ee-c1274e3a634d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.105537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4f637-da88-4a5e-8c1f-c4d4cef87e68 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.107006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb4f637-da88-4a5e-8c1f-c4d4cef87e68 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.154522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebfd4277-4479-43f2-b8ee-c1274e3a634d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.186864 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.194425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed29ab28-7083-4de1-91e4-862879b0ac3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.195246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29ab28-7083-4de1-91e4-862879b0ac3c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.195529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29ab28-7083-4de1-91e4-862879b0ac3c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.195802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed29ab28-7083-4de1-91e4-862879b0ac3c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.196459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29ab28-7083-4de1-91e4-862879b0ac3c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.197957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed29ab28-7083-4de1-91e4-862879b0ac3c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.211456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 346/711] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 441 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ed29ab28-7083-4de1-91e4-862879b0ac3c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:41.212409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:41.213317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:10:41.214193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:41.216545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33085a2-ffef-4294-a699-aa96eb439aeb None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.216545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33085a2-ffef-4294-a699-aa96eb439aeb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.216545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33085a2-ffef-4294-a699-aa96eb439aeb None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.216991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 366/712] 10.222.0.69 () {56 vars in 1129 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:41.217132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 347/713] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:41.220979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33085a2-ffef-4294-a699-aa96eb439aeb None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.223761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5584a-223f-4fcd-8a25-e5ef5febb275 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.223761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5584a-223f-4fcd-8a25-e5ef5febb275 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.223761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda5584a-223f-4fcd-8a25-e5ef5febb275 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.241146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bda5584a-223f-4fcd-8a25-e5ef5febb275 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:41.242379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bda5584a-223f-4fcd-8a25-e5ef5febb275 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:41.273724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.331881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33085a2-ffef-4294-a699-aa96eb439aeb None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.337638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 367/714] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:41.338562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5584a-223f-4fcd-8a25-e5ef5febb275 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.368454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f52ae-4f2f-4b20-9531-9733cfb35da6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.369372 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f52ae-4f2f-4b20-9531-9733cfb35da6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.369831 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f52ae-4f2f-4b20-9531-9733cfb35da6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.370356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc0f52ae-4f2f-4b20-9531-9733cfb35da6 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.371927 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f52ae-4f2f-4b20-9531-9733cfb35da6 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:41.372381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f52ae-4f2f-4b20-9531-9733cfb35da6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.373999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc0f52ae-4f2f-4b20-9531-9733cfb35da6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.386474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 348/715] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:41.387776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f52ae-4f2f-4b20-9531-9733cfb35da6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.432142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.470543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda5584a-223f-4fcd-8a25-e5ef5febb275 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.499357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc0f52ae-4f2f-4b20-9531-9733cfb35da6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.509470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdcb48-f10b-4d64-bbc5-ec9bb87a232e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.510384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdcb48-f10b-4d64-bbc5-ec9bb87a232e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.510729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdcb48-f10b-4d64-bbc5-ec9bb87a232e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.511088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccdcb48-f10b-4d64-bbc5-ec9bb87a232e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.522590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccdcb48-f10b-4d64-bbc5-ec9bb87a232e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.524141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccdcb48-f10b-4d64-bbc5-ec9bb87a232e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.542366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba8c55-d31f-4163-8764-275befc6a41d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.544046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba8c55-d31f-4163-8764-275befc6a41d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-cccdcb48-f10b-4d64-bbc5-ec9bb87a232e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:41.544159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:41.545015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:41.546024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:41.546024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:41.546024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:41.546024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:41.547052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba8c55-d31f-4163-8764-275befc6a41d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.547532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ba8c55-d31f-4163-8764-275befc6a41d None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.548439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba8c55-d31f-4163-8764-275befc6a41d None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:41.548944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba8c55-d31f-4163-8764-275befc6a41d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.550346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 368/716] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:41.550946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ba8c55-d31f-4163-8764-275befc6a41d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.551232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdcb48-f10b-4d64-bbc5-ec9bb87a232e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.559639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 349/717] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:41.560983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba8c55-d31f-4163-8764-275befc6a41d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.573588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.681710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccdcb48-f10b-4d64-bbc5-ec9bb87a232e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.719718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72324371-ffe0-4df8-9f77-0f66c6071c1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.720688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72324371-ffe0-4df8-9f77-0f66c6071c1e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.721072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72324371-ffe0-4df8-9f77-0f66c6071c1e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.721450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72324371-ffe0-4df8-9f77-0f66c6071c1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.733717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72324371-ffe0-4df8-9f77-0f66c6071c1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-72324371-ffe0-4df8-9f77-0f66c6071c1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:41.741542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:41.742519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:41.743796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 369/718] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:41.750550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72324371-ffe0-4df8-9f77-0f66c6071c1e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.757700 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ba8c55-d31f-4163-8764-275befc6a41d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.765331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.816179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998aa190-a77a-4b6c-b7b4-034edfdc5a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:41.816906 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aa190-a77a-4b6c-b7b4-034edfdc5a64 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:41.817186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aa190-a77a-4b6c-b7b4-034edfdc5a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:41.817522 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-998aa190-a77a-4b6c-b7b4-034edfdc5a64 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:41.818252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998aa190-a77a-4b6c-b7b4-034edfdc5a64 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:41.818556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998aa190-a77a-4b6c-b7b4-034edfdc5a64 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:41.819922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-998aa190-a77a-4b6c-b7b4-034edfdc5a64 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:41.849960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 350/719] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:41.862554 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998aa190-a77a-4b6c-b7b4-034edfdc5a64 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:41.875089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:41.924751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72324371-ffe0-4df8-9f77-0f66c6071c1e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:41.982377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-998aa190-a77a-4b6c-b7b4-034edfdc5a64 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:42.006405 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62ea224f-f24b-4032-9fe9-3af552a13bc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:42.007810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea224f-f24b-4032-9fe9-3af552a13bc8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.008418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea224f-f24b-4032-9fe9-3af552a13bc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.008974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62ea224f-f24b-4032-9fe9-3af552a13bc8 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.010132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea224f-f24b-4032-9fe9-3af552a13bc8 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:42.010932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea224f-f24b-4032-9fe9-3af552a13bc8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:42.013182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62ea224f-f24b-4032-9fe9-3af552a13bc8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:42.018348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039449e4-23e1-4788-9096-9f51cda2c6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:42.019222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039449e4-23e1-4788-9096-9f51cda2c6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.019580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039449e4-23e1-4788-9096-9f51cda2c6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.020187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039449e4-23e1-4788-9096-9f51cda2c6e3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.024111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 370/720] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:42.025751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec991b-41c2-4774-b39b-ea982d62e0aa None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.026498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec991b-41c2-4774-b39b-ea982d62e0aa None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.027009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fec991b-41c2-4774-b39b-ea982d62e0aa None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.027116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039449e4-23e1-4788-9096-9f51cda2c6e3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:42.028686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039449e4-23e1-4788-9096-9f51cda2c6e3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:42.033604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 371/721] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:42.035228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec991b-41c2-4774-b39b-ea982d62e0aa None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-039449e4-23e1-4788-9096-9f51cda2c6e3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:42.047609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:42.048544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:42.049638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:42.049638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:42.049638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:42.049638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:42.049638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:42.049638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:42.049638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:42.049638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:42.049638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 351/722] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:42.051043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef7a80-3e50-4e8a-9112-6f87d1d831b5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.051403 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef7a80-3e50-4e8a-9112-6f87d1d831b5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.051842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ef7a80-3e50-4e8a-9112-6f87d1d831b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.072015 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-59ef7a80-3e50-4e8a-9112-6f87d1d831b5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:42.073615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-59ef7a80-3e50-4e8a-9112-6f87d1d831b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:42.100680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:42.165862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 352/723] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:42.165862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef7a80-3e50-4e8a-9112-6f87d1d831b5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:42.182799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fec991b-41c2-4774-b39b-ea982d62e0aa None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:42.230790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e6078-45a3-4063-b73b-ef20796290c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:42.231911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e6078-45a3-4063-b73b-ef20796290c8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.232343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e6078-45a3-4063-b73b-ef20796290c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.232798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3e6078-45a3-4063-b73b-ef20796290c8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.241988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:42.247077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e6078-45a3-4063-b73b-ef20796290c8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:42.250360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3e6078-45a3-4063-b73b-ef20796290c8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-2a3e6078-45a3-4063-b73b-ef20796290c8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:42.272938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:42.274060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:42.274979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:42.274979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:42.274979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:42.274979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:42.274979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 372/724] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:42.275825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e6078-45a3-4063-b73b-ef20796290c8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:42.308030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ef7a80-3e50-4e8a-9112-6f87d1d831b5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:42.353332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a860a3f-0b63-45a4-9819-57e61b71b997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:42.354072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a860a3f-0b63-45a4-9819-57e61b71b997 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.354364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a860a3f-0b63-45a4-9819-57e61b71b997 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.354647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a860a3f-0b63-45a4-9819-57e61b71b997 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.356052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a860a3f-0b63-45a4-9819-57e61b71b997 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:42.356327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a860a3f-0b63-45a4-9819-57e61b71b997 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:42.357807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a860a3f-0b63-45a4-9819-57e61b71b997 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:42.370388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 353/725] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:42.371398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a860a3f-0b63-45a4-9819-57e61b71b997 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:42.391804 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:42.430809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3e6078-45a3-4063-b73b-ef20796290c8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:42.478115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae04694-6ff6-4f47-bf95-dad478c0d489 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:42.479381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae04694-6ff6-4f47-bf95-dad478c0d489 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.479838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae04694-6ff6-4f47-bf95-dad478c0d489 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.481189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae04694-6ff6-4f47-bf95-dad478c0d489 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.483645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a860a3f-0b63-45a4-9819-57e61b71b997 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:42.495046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae04694-6ff6-4f47-bf95-dad478c0d489 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-1ae04694-6ff6-4f47-bf95-dad478c0d489 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:42.505359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:42.506467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:42.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:42.515820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 373/726] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:42.515820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae04694-6ff6-4f47-bf95-dad478c0d489 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:42.531303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3357c9-9539-422b-b4c4-ce9a72e56cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:42.532535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3357c9-9539-422b-b4c4-ce9a72e56cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.533053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3357c9-9539-422b-b4c4-ce9a72e56cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.533802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b3357c9-9539-422b-b4c4-ce9a72e56cd8 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.535597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3357c9-9539-422b-b4c4-ce9a72e56cd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:42.536403 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3357c9-9539-422b-b4c4-ce9a72e56cd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:42.538669 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b3357c9-9539-422b-b4c4-ce9a72e56cd8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:42.556974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 354/727] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:42.558667 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3357c9-9539-422b-b4c4-ce9a72e56cd8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:42.574476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:42.631310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae04694-6ff6-4f47-bf95-dad478c0d489 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:42.670201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:42.674408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bee4c-43a1-4bf8-ac80-921843df001f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:42.675216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bee4c-43a1-4bf8-ac80-921843df001f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.675506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bee4c-43a1-4bf8-ac80-921843df001f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.675786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4bee4c-43a1-4bf8-ac80-921843df001f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.676622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bee4c-43a1-4bf8-ac80-921843df001f None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=36955d7d4505457696ab6177adb999a9) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:42.676939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bee4c-43a1-4bf8-ac80-921843df001f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:42.678972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4bee4c-43a1-4bf8-ac80-921843df001f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:42.687905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 374/728] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:42 2022] GET /identity/v3/role_assignments?user.id=36955d7d4505457696ab6177adb999a9&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:42.689112 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bee4c-43a1-4bf8-ac80-921843df001f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:42.703417 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b3357c9-9539-422b-b4c4-ce9a72e56cd8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:42.767007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76ffa5-4117-4aeb-a568-d93d2f7c0c49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:42.768142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76ffa5-4117-4aeb-a568-d93d2f7c0c49 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.768642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76ffa5-4117-4aeb-a568-d93d2f7c0c49 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.769155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e76ffa5-4117-4aeb-a568-d93d2f7c0c49 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.771525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76ffa5-4117-4aeb-a568-d93d2f7c0c49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:42.772276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76ffa5-4117-4aeb-a568-d93d2f7c0c49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:42.776111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e76ffa5-4117-4aeb-a568-d93d2f7c0c49 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:42.796709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 355/729] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:42.807206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76ffa5-4117-4aeb-a568-d93d2f7c0c49 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:42.877444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4bee4c-43a1-4bf8-ac80-921843df001f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:42.896030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:42.930996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b01dc-a9f1-41d1-9781-82acfd9aaeb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:42.932181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b01dc-a9f1-41d1-9781-82acfd9aaeb8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:42.933887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b01dc-a9f1-41d1-9781-82acfd9aaeb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:42.934759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-640b01dc-a9f1-41d1-9781-82acfd9aaeb8 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:42.946427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b01dc-a9f1-41d1-9781-82acfd9aaeb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:42.948229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-640b01dc-a9f1-41d1-9781-82acfd9aaeb8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-640b01dc-a9f1-41d1-9781-82acfd9aaeb8 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:42.975179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:42.976212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:42.977167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:42.977167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:42.977167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:42.977167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:42.977167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:42.977167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:42.977167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:42.977167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:42.977167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:10:42.977167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:42.984999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 375/730] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Jul 20 19:10:42 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 296 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:42.991018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b01dc-a9f1-41d1-9781-82acfd9aaeb8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:43.015011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e76ffa5-4117-4aeb-a568-d93d2f7c0c49 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:43.051177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993dab2-f583-4360-87cc-7e85f1abaeb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:43.051895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993dab2-f583-4360-87cc-7e85f1abaeb0 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:43.052179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993dab2-f583-4360-87cc-7e85f1abaeb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:43.052458 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5993dab2-f583-4360-87cc-7e85f1abaeb0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:43.069913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:43.075614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993dab2-f583-4360-87cc-7e85f1abaeb0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:43.077088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5993dab2-f583-4360-87cc-7e85f1abaeb0 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-5993dab2-f583-4360-87cc-7e85f1abaeb0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:43.111268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:43.112212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:43.113121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:43.113121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:43.113121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:43.113121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:43.117534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 356/731] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 307 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:43.123206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993dab2-f583-4360-87cc-7e85f1abaeb0 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:43.178675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-640b01dc-a9f1-41d1-9781-82acfd9aaeb8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:43.230525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5993dab2-f583-4360-87cc-7e85f1abaeb0 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:43.241356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69c19b-4c10-464c-911e-1174254425c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:43.242382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69c19b-4c10-464c-911e-1174254425c9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:43.242869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69c19b-4c10-464c-911e-1174254425c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:43.243375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e69c19b-4c10-464c-911e-1174254425c9 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:43.244427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69c19b-4c10-464c-911e-1174254425c9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:43.244939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69c19b-4c10-464c-911e-1174254425c9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:43.247453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e69c19b-4c10-464c-911e-1174254425c9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:43.277140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaaef0fe-deb0-4765-97b5-b4c62b0d412a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:43.278517 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaef0fe-deb0-4765-97b5-b4c62b0d412a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:43.278808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaef0fe-deb0-4765-97b5-b4c62b0d412a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:43.279090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaaef0fe-deb0-4765-97b5-b4c62b0d412a None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:43.280063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaef0fe-deb0-4765-97b5-b4c62b0d412a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:43.280376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaef0fe-deb0-4765-97b5-b4c62b0d412a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:43.283175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 376/732] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Jul 20 19:10:42 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:43.283889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaaef0fe-deb0-4765-97b5-b4c62b0d412a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:43.288669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69c19b-4c10-464c-911e-1174254425c9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:43.314096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 357/733] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:43.316464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52d4fa-7a1b-43ad-a32c-8f3abb8b8920 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:43.316464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52d4fa-7a1b-43ad-a32c-8f3abb8b8920 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:43.316464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d52d4fa-7a1b-43ad-a32c-8f3abb8b8920 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:43.317305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 358/734] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:43.321175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52d4fa-7a1b-43ad-a32c-8f3abb8b8920 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:43.352078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:43.379102 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:43.438600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d52d4fa-7a1b-43ad-a32c-8f3abb8b8920 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:43.445470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e69c19b-4c10-464c-911e-1174254425c9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:43.476676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1607151-d33b-4cf1-99a8-b915fdaa004b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:43.477744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607151-d33b-4cf1-99a8-b915fdaa004b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:43.478276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607151-d33b-4cf1-99a8-b915fdaa004b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:43.478838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1607151-d33b-4cf1-99a8-b915fdaa004b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:43.488397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1607151-d33b-4cf1-99a8-b915fdaa004b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:43.489536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc11b8-735b-41b8-abdd-eeed52cad2bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:43.490319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc11b8-735b-41b8-abdd-eeed52cad2bb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:43.490676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1607151-d33b-4cf1-99a8-b915fdaa004b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:43.490790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc11b8-735b-41b8-abdd-eeed52cad2bb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:43.491138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfc11b8-735b-41b8-abdd-eeed52cad2bb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-b1607151-d33b-4cf1-99a8-b915fdaa004b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:43.507794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:43.508979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:43.510130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:43.510130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:43.510130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:43.510130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:43.510130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:43.510130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:43.510130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:43.510130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:43.510130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:43.510130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:43.516972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 359/735] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:43 2022] GET /identity/v3/users/swift => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:43.518817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034be9e-ae8f-4cba-99f4-f17f86ddd53f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:43.519237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034be9e-ae8f-4cba-99f4-f17f86ddd53f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:43.519655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034be9e-ae8f-4cba-99f4-f17f86ddd53f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:43.538670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6034be9e-ae8f-4cba-99f4-f17f86ddd53f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:43.540231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6034be9e-ae8f-4cba-99f4-f17f86ddd53f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:43.578130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc11b8-735b-41b8-abdd-eeed52cad2bb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:43.579977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfc11b8-735b-41b8-abdd-eeed52cad2bb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:43.613913 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:43.629645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 360/736] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:43.630743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034be9e-ae8f-4cba-99f4-f17f86ddd53f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:43.678863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:43.739983 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 377/737] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 451 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:43.740919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc11b8-735b-41b8-abdd-eeed52cad2bb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:43.747183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034be9e-ae8f-4cba-99f4-f17f86ddd53f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:43.784078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf51c59-2cb9-4df2-9e94-363a0cebcf32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:43.784751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51c59-2cb9-4df2-9e94-363a0cebcf32 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:43.785029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51c59-2cb9-4df2-9e94-363a0cebcf32 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:43.785298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf51c59-2cb9-4df2-9e94-363a0cebcf32 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:43.786297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf51c59-2cb9-4df2-9e94-363a0cebcf32 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:43.786601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf51c59-2cb9-4df2-9e94-363a0cebcf32 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:43.788027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adf51c59-2cb9-4df2-9e94-363a0cebcf32 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:43.809331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 361/738] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Jul 20 19:10:43 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:43.810671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba13779-2c63-4307-8d13-9e67413fd02c None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:43.810958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba13779-2c63-4307-8d13-9e67413fd02c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:43.811235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba13779-2c63-4307-8d13-9e67413fd02c None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:43.812630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 362/739] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:43.817118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba13779-2c63-4307-8d13-9e67413fd02c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:43.851501 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfc11b8-735b-41b8-abdd-eeed52cad2bb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:43.883991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:43.890594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880a0e4-3181-402b-a5ea-8e2b0dd29ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:43.891261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880a0e4-3181-402b-a5ea-8e2b0dd29ae0 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:43.891543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880a0e4-3181-402b-a5ea-8e2b0dd29ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:43.891846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d880a0e4-3181-402b-a5ea-8e2b0dd29ae0 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:43.892537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880a0e4-3181-402b-a5ea-8e2b0dd29ae0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:43.893932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d880a0e4-3181-402b-a5ea-8e2b0dd29ae0 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:43.910013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 378/740] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:10:43 2022] POST /identity/v3/services => generated 240 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:43.910840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880a0e4-3181-402b-a5ea-8e2b0dd29ae0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:43.931242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:43.947117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba13779-2c63-4307-8d13-9e67413fd02c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.001739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c91604-fbd6-43b6-a102-7f0a42e9ceae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.002617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c91604-fbd6-43b6-a102-7f0a42e9ceae None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.003232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c91604-fbd6-43b6-a102-7f0a42e9ceae None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.003516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c91604-fbd6-43b6-a102-7f0a42e9ceae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.019594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c91604-fbd6-43b6-a102-7f0a42e9ceae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.021140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c91604-fbd6-43b6-a102-7f0a42e9ceae None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-92c91604-fbd6-43b6-a102-7f0a42e9ceae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:44.043605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:44.044588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:44.045504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:44.045504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:44.045504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:44.045504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:44.047887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 363/741] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:44.049551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e9bcc-3a83-4ed5-9878-4dcb6c13311d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.050166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e9bcc-3a83-4ed5-9878-4dcb6c13311d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.050690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e9bcc-3a83-4ed5-9878-4dcb6c13311d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.065311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d880a0e4-3181-402b-a5ea-8e2b0dd29ae0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.068844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-252e9bcc-3a83-4ed5-9878-4dcb6c13311d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:44.070165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-252e9bcc-3a83-4ed5-9878-4dcb6c13311d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:44.108016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b61e2-9816-4e2f-8750-c08457966174 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.108599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b61e2-9816-4e2f-8750-c08457966174 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.108885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b61e2-9816-4e2f-8750-c08457966174 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.109164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89b61e2-9816-4e2f-8750-c08457966174 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.160389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 364/742] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:44 2022] POST /identity/v3/auth/tokens => generated 898 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 20 19:10:44.161554 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e9bcc-3a83-4ed5-9878-4dcb6c13311d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.200518 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:44.218160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b61e2-9816-4e2f-8750-c08457966174 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.220293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89b61e2-9816-4e2f-8750-c08457966174 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.249314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:44.275866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e9bcc-3a83-4ed5-9878-4dcb6c13311d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.314568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03f653-aade-48fb-a844-98a6aadbae57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.315542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03f653-aade-48fb-a844-98a6aadbae57 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.316078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03f653-aade-48fb-a844-98a6aadbae57 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.316585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd03f653-aade-48fb-a844-98a6aadbae57 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.317627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03f653-aade-48fb-a844-98a6aadbae57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:44.318086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03f653-aade-48fb-a844-98a6aadbae57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.319726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd03f653-aade-48fb-a844-98a6aadbae57 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.339084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 365/743] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:44.340687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03f653-aade-48fb-a844-98a6aadbae57 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.360270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 379/744] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:43 2022] GET /identity/v3/auth/tokens => generated 898 bytes in 450 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 20 19:10:44.361457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b61e2-9816-4e2f-8750-c08457966174 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.469002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:44.478311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd03f653-aade-48fb-a844-98a6aadbae57 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.481051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89b61e2-9816-4e2f-8750-c08457966174 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.516205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0656edc-c5d4-46e0-bc70-c50d0a5b5a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.517144 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0656edc-c5d4-46e0-bc70-c50d0a5b5a01 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.517766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0656edc-c5d4-46e0-bc70-c50d0a5b5a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.518287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0656edc-c5d4-46e0-bc70-c50d0a5b5a01 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.519153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0656edc-c5d4-46e0-bc70-c50d0a5b5a01 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.520690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0656edc-c5d4-46e0-bc70-c50d0a5b5a01 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.523194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e74253-4af4-44c1-b726-4367c697ad76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.524007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e74253-4af4-44c1-b726-4367c697ad76 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.524295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e74253-4af4-44c1-b726-4367c697ad76 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.524574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e74253-4af4-44c1-b726-4367c697ad76 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.525689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:44.534899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e74253-4af4-44c1-b726-4367c697ad76 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.536412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e74253-4af4-44c1-b726-4367c697ad76 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.542357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 366/745] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:10:44 2022] POST /identity/v3/services => generated 251 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:44.544173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0656edc-c5d4-46e0-bc70-c50d0a5b5a01 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-03e74253-4af4-44c1-b726-4367c697ad76 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:44.563731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:44.564927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:44.565976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:44.565976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:44.565976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:44.565976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:44.565976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:44.565976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:44.565976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:44.565976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:44.565976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 380/746] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:44.582999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e74253-4af4-44c1-b726-4367c697ad76 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.675002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0656edc-c5d4-46e0-bc70-c50d0a5b5a01 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.721439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32fb4e-5248-4295-8357-eaac48e5c886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.722557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32fb4e-5248-4295-8357-eaac48e5c886 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.722986 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32fb4e-5248-4295-8357-eaac48e5c886 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.723685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea32fb4e-5248-4295-8357-eaac48e5c886 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.752927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e74253-4af4-44c1-b726-4367c697ad76 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.754077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32fb4e-5248-4295-8357-eaac48e5c886 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.755710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea32fb4e-5248-4295-8357-eaac48e5c886 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-ea32fb4e-5248-4295-8357-eaac48e5c886 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:44.791725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:44.793855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:44.795057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:44.795057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:44.795057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:44.795057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:44.796736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 367/747] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:44.796925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58c3dc-f536-47f1-aaac-461e40eeadef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:44.796925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58c3dc-f536-47f1-aaac-461e40eeadef None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:44.797169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58c3dc-f536-47f1-aaac-461e40eeadef None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:44.797817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a58c3dc-f536-47f1-aaac-461e40eeadef None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:44.798334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58c3dc-f536-47f1-aaac-461e40eeadef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:44.798648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58c3dc-f536-47f1-aaac-461e40eeadef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:44.800122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a58c3dc-f536-47f1-aaac-461e40eeadef None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:44.802533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32fb4e-5248-4295-8357-eaac48e5c886 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.833592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 381/748] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:44.837098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58c3dc-f536-47f1-aaac-461e40eeadef None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:44.863540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:44.882747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:44.959257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea32fb4e-5248-4295-8357-eaac48e5c886 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:44.980611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a58c3dc-f536-47f1-aaac-461e40eeadef None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.001198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f11f385e-a873-4ab7-a19a-5a1fcd46f991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.001918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f385e-a873-4ab7-a19a-5a1fcd46f991 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.002201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f385e-a873-4ab7-a19a-5a1fcd46f991 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.002475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f11f385e-a873-4ab7-a19a-5a1fcd46f991 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.003195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f385e-a873-4ab7-a19a-5a1fcd46f991 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:45.003503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f385e-a873-4ab7-a19a-5a1fcd46f991 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.004875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f11f385e-a873-4ab7-a19a-5a1fcd46f991 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.021007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 368/749] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:45.023502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c380d-38b1-411c-bbe6-572ce21dcb2d None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.023940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c380d-38b1-411c-bbe6-572ce21dcb2d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.024388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063c380d-38b1-411c-bbe6-572ce21dcb2d None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.027860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 369/750] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:45 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:45.028637 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85449f4-fb0e-4a7e-b8d9-be48fad119f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.029612 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85449f4-fb0e-4a7e-b8d9-be48fad119f6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.030050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85449f4-fb0e-4a7e-b8d9-be48fad119f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.030494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85449f4-fb0e-4a7e-b8d9-be48fad119f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.031485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85449f4-fb0e-4a7e-b8d9-be48fad119f6 None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=cbd4016436b54a0fb8c74c35273957a0) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:45.032368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85449f4-fb0e-4a7e-b8d9-be48fad119f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.033843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c85449f4-fb0e-4a7e-b8d9-be48fad119f6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.034483 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c380d-38b1-411c-bbe6-572ce21dcb2d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.040342 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 382/751] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:44 2022] GET /identity/v3/role_assignments?user.id=cbd4016436b54a0fb8c74c35273957a0&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:45.049370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f32261-40cb-4c36-b606-ed3a56fbf030 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.049814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f32261-40cb-4c36-b606-ed3a56fbf030 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.050107 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2f32261-40cb-4c36-b606-ed3a56fbf030 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.094862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b2f32261-40cb-4c36-b606-ed3a56fbf030 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:45.097286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b2f32261-40cb-4c36-b606-ed3a56fbf030 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:45.195090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063c380d-38b1-411c-bbe6-572ce21dcb2d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.211480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.252669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 383/752] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:45 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:45.259759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103c435-9313-4d95-854d-f44d7f210fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.260555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103c435-9313-4d95-854d-f44d7f210fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.260837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103c435-9313-4d95-854d-f44d7f210fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.261173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f103c435-9313-4d95-854d-f44d7f210fe2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.269454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103c435-9313-4d95-854d-f44d7f210fe2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.270936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f103c435-9313-4d95-854d-f44d7f210fe2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.280986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f32261-40cb-4c36-b606-ed3a56fbf030 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f103c435-9313-4d95-854d-f44d7f210fe2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:45.285492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:45.286481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:45.287415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:45.287415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:45.287415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:45.287415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:45.287415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:45.287415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:45.287415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:45.287415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:45.287415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 370/753] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 255 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:45.293997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.294813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103c435-9313-4d95-854d-f44d7f210fe2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.468683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2f32261-40cb-4c36-b606-ed3a56fbf030 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.483578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f103c435-9313-4d95-854d-f44d7f210fe2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.531501 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c32cd-e185-411a-b660-196682a43dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.532480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c32cd-e185-411a-b660-196682a43dcc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.532951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c32cd-e185-411a-b660-196682a43dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.533411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423c32cd-e185-411a-b660-196682a43dcc None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.534540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c32cd-e185-411a-b660-196682a43dcc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:45.535035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c32cd-e185-411a-b660-196682a43dcc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.536673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423c32cd-e185-411a-b660-196682a43dcc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.547636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8302d1-ac1f-4279-b6c7-c463740c177a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.548448 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8302d1-ac1f-4279-b6c7-c463740c177a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.548733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8302d1-ac1f-4279-b6c7-c463740c177a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.549010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8302d1-ac1f-4279-b6c7-c463740c177a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.554919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 371/754] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:45.561990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c32cd-e185-411a-b660-196682a43dcc None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.580743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8302d1-ac1f-4279-b6c7-c463740c177a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ac8302d1-ac1f-4279-b6c7-c463740c177a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:45.588838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:45.589855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:45.590720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:45.597607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 384/755] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 313 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:45.602485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8302d1-ac1f-4279-b6c7-c463740c177a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.619594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.722734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.762275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423c32cd-e185-411a-b660-196682a43dcc None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.788771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8302d1-ac1f-4279-b6c7-c463740c177a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:45.823192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b35e2-30cd-41e3-8be1-bdb0aa0bc877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.824798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b35e2-30cd-41e3-8be1-bdb0aa0bc877 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.825089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b35e2-30cd-41e3-8be1-bdb0aa0bc877 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.825372 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0b35e2-30cd-41e3-8be1-bdb0aa0bc877 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.826239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b35e2-30cd-41e3-8be1-bdb0aa0bc877 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=9d8387328e4145db80782f2d54c0966d) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:45.827183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b35e2-30cd-41e3-8be1-bdb0aa0bc877 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.828654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0b35e2-30cd-41e3-8be1-bdb0aa0bc877 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.839855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 372/756] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:45 2022] GET /identity/v3/role_assignments?user.id=9d8387328e4145db80782f2d54c0966d&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 259 bytes in 277 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:45.848744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dc94c-4ba5-4bb9-a8ba-b529e37a6396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:45.849637 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dc94c-4ba5-4bb9-a8ba-b529e37a6396 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:45.850025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dc94c-4ba5-4bb9-a8ba-b529e37a6396 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:45.850403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022dc94c-4ba5-4bb9-a8ba-b529e37a6396 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:45.851273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022dc94c-4ba5-4bb9-a8ba-b529e37a6396 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:45.851688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022dc94c-4ba5-4bb9-a8ba-b529e37a6396 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:45.853177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022dc94c-4ba5-4bb9-a8ba-b529e37a6396 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:45.864061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 385/757] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:45.869338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b35e2-30cd-41e3-8be1-bdb0aa0bc877 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:45.985397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:45.999966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0b35e2-30cd-41e3-8be1-bdb0aa0bc877 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:46.036811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8650e671-84ff-4048-9c39-9ce60b924eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:46.037678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e671-84ff-4048-9c39-9ce60b924eeb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.038252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e671-84ff-4048-9c39-9ce60b924eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.038533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8650e671-84ff-4048-9c39-9ce60b924eeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.050575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e671-84ff-4048-9c39-9ce60b924eeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:46.052063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8650e671-84ff-4048-9c39-9ce60b924eeb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8650e671-84ff-4048-9c39-9ce60b924eeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:46.072169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:46.073128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:46.075310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:46.075310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:46.075310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:46.075310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:46.075310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 373/758] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:46.080247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dc94c-4ba5-4bb9-a8ba-b529e37a6396 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:46.232648 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213e68d-cfca-4f7d-acd4-a67663605c4e None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.233020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213e68d-cfca-4f7d-acd4-a67663605c4e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.233213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b213e68d-cfca-4f7d-acd4-a67663605c4e None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.234807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 374/759] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:46.240033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926abd6c-37a9-41d6-8289-6b9f9eea2ac8 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.240296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926abd6c-37a9-41d6-8289-6b9f9eea2ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.240524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926abd6c-37a9-41d6-8289-6b9f9eea2ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.289492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-926abd6c-37a9-41d6-8289-6b9f9eea2ac8 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:46.290729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-926abd6c-37a9-41d6-8289-6b9f9eea2ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:46.313095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022dc94c-4ba5-4bb9-a8ba-b529e37a6396 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:46.346667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:46.360344 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32ebc4-d9d3-40b3-a595-0ee1a0d02916 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:46.362358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32ebc4-d9d3-40b3-a595-0ee1a0d02916 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.362866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32ebc4-d9d3-40b3-a595-0ee1a0d02916 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.363498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed32ebc4-d9d3-40b3-a595-0ee1a0d02916 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.365424 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32ebc4-d9d3-40b3-a595-0ee1a0d02916 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:46.366014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32ebc4-d9d3-40b3-a595-0ee1a0d02916 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:46.367694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed32ebc4-d9d3-40b3-a595-0ee1a0d02916 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:46.387592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 386/760] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:46.393229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32ebc4-d9d3-40b3-a595-0ee1a0d02916 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:46.460207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 375/761] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 221 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:46.498325 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926abd6c-37a9-41d6-8289-6b9f9eea2ac8 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:46.565227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed32ebc4-d9d3-40b3-a595-0ee1a0d02916 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:46.596279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:46.642410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926abd6c-37a9-41d6-8289-6b9f9eea2ac8 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:46.645590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc993f9-192c-4bc3-af79-145cff974ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:46.646740 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc993f9-192c-4bc3-af79-145cff974ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.647214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc993f9-192c-4bc3-af79-145cff974ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.647753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc993f9-192c-4bc3-af79-145cff974ff6 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.662800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc993f9-192c-4bc3-af79-145cff974ff6 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:46.673777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc993f9-192c-4bc3-af79-145cff974ff6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:46.684201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7eca2-d169-43c9-b7eb-e7df14e124c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:46.685024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7eca2-d169-43c9-b7eb-e7df14e124c1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.685314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7eca2-d169-43c9-b7eb-e7df14e124c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.685644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f7eca2-d169-43c9-b7eb-e7df14e124c1 None admin] PATH_INFO: `/v3/services/network` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.686308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7eca2-d169-43c9-b7eb-e7df14e124c1 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:46.687790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f7eca2-d169-43c9-b7eb-e7df14e124c1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-57f7eca2-d169-43c9-b7eb-e7df14e124c1 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:46.702664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:46.703707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:46.704797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 376/762] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:46 2022] GET /identity/v3/services/network => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ebc993f9-192c-4bc3-af79-145cff974ff6 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:46.709054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:46.710027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:46.712559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:46.712559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:46.712559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:46.712559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:46.712559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:46.712559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:46.712559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:46.712559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:46.712559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:46.712559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:46.713845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7eca2-d169-43c9-b7eb-e7df14e124c1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:46.713992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 387/763] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Jul 20 19:10:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:46.718474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc993f9-192c-4bc3-af79-145cff974ff6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:46.818687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f7eca2-d169-43c9-b7eb-e7df14e124c1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:46.844214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:46.878790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb65b6-2702-49b6-83b6-f9edac9064ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:46.879470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb65b6-2702-49b6-83b6-f9edac9064ed None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.879759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb65b6-2702-49b6-83b6-f9edac9064ed None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.880041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baeb65b6-2702-49b6-83b6-f9edac9064ed None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.880758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb65b6-2702-49b6-83b6-f9edac9064ed None admin] RBAC: Adding query filter params (name=network) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:46.881068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb65b6-2702-49b6-83b6-f9edac9064ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:46.882496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baeb65b6-2702-49b6-83b6-f9edac9064ed None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:46.889829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:46.897629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 377/764] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:10:46 2022] GET /identity/v3/services?name=network => generated 124 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:46.904366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb65b6-2702-49b6-83b6-f9edac9064ed None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:46.942623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc993f9-192c-4bc3-af79-145cff974ff6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:46.989979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93767306-fcdc-47eb-b222-f138d2c37df4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:46.991039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93767306-fcdc-47eb-b222-f138d2c37df4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:46.991326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93767306-fcdc-47eb-b222-f138d2c37df4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:46.991602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93767306-fcdc-47eb-b222-f138d2c37df4 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:46.992442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93767306-fcdc-47eb-b222-f138d2c37df4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:46.992753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93767306-fcdc-47eb-b222-f138d2c37df4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:47.001420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93767306-fcdc-47eb-b222-f138d2c37df4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:47.039835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 388/765] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Jul 20 19:10:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:47.045206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93767306-fcdc-47eb-b222-f138d2c37df4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:47.083732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baeb65b6-2702-49b6-83b6-f9edac9064ed None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:47.121547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e327a0d-e099-4079-81e6-6ccee59f2b1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:47.122333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e327a0d-e099-4079-81e6-6ccee59f2b1a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.122625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e327a0d-e099-4079-81e6-6ccee59f2b1a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.122968 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e327a0d-e099-4079-81e6-6ccee59f2b1a None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.123679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e327a0d-e099-4079-81e6-6ccee59f2b1a None admin] RBAC: Adding query filter params (type=network) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:47.124038 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e327a0d-e099-4079-81e6-6ccee59f2b1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:47.125435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e327a0d-e099-4079-81e6-6ccee59f2b1a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:47.131211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 378/766] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:10:46 2022] GET /identity/v3/services?type=network => generated 350 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:47.139194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e327a0d-e099-4079-81e6-6ccee59f2b1a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:47.238182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93767306-fcdc-47eb-b222-f138d2c37df4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:47.255545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:47.271125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:47.304141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c272041-bf58-4d8e-920b-c77c97f634c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:47.305360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c272041-bf58-4d8e-920b-c77c97f634c3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.305673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c272041-bf58-4d8e-920b-c77c97f634c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.305951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c272041-bf58-4d8e-920b-c77c97f634c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.327887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c272041-bf58-4d8e-920b-c77c97f634c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:47.329845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c272041-bf58-4d8e-920b-c77c97f634c3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-4c272041-bf58-4d8e-920b-c77c97f634c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:47.351625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:47.352632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:47.354105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:47.354105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:47.354105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:47.354105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:47.354105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 389/767] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:47.359808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c272041-bf58-4d8e-920b-c77c97f634c3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:47.367638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e327a0d-e099-4079-81e6-6ccee59f2b1a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:47.406216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edb6ff8-2c76-42ea-9d62-ae359feb697c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:47.406890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb6ff8-2c76-42ea-9d62-ae359feb697c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.407224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb6ff8-2c76-42ea-9d62-ae359feb697c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.407508 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edb6ff8-2c76-42ea-9d62-ae359feb697c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.408307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb6ff8-2c76-42ea-9d62-ae359feb697c None admin] RBAC: Adding query filter params (interface=public, service_id=8ab257c2ba3d4ae89f346631a207302b, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:47.408624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb6ff8-2c76-42ea-9d62-ae359feb697c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:47.410059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6edb6ff8-2c76-42ea-9d62-ae359feb697c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:47.419504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 379/768] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/endpoints?service_id=8ab257c2ba3d4ae89f346631a207302b&interface=public®ion_id=RegionOne => generated 194 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:47.420680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c96287-0d28-4267-bffb-9047d822fe5e None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.420977 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c96287-0d28-4267-bffb-9047d822fe5e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.421259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c96287-0d28-4267-bffb-9047d822fe5e None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.427077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 380/769] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:47.427940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c96287-0d28-4267-bffb-9047d822fe5e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:47.487235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c272041-bf58-4d8e-920b-c77c97f634c3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:47.522583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1cd111-d952-4078-a98f-abffe1469917 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:47.523237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cd111-d952-4078-a98f-abffe1469917 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.523517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cd111-d952-4078-a98f-abffe1469917 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.523791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1cd111-d952-4078-a98f-abffe1469917 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.524497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cd111-d952-4078-a98f-abffe1469917 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:47.524802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cd111-d952-4078-a98f-abffe1469917 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:47.526176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1cd111-d952-4078-a98f-abffe1469917 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:47.539384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 390/770] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:47.544324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c96287-0d28-4267-bffb-9047d822fe5e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:47.546052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78eb8d7-e84c-48d2-bcdd-640e6f5aeda3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.546343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78eb8d7-e84c-48d2-bcdd-640e6f5aeda3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.546615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78eb8d7-e84c-48d2-bcdd-640e6f5aeda3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.564156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b78eb8d7-e84c-48d2-bcdd-640e6f5aeda3 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:47.565354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b78eb8d7-e84c-48d2-bcdd-640e6f5aeda3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:47.572339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:47.580243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:47.585664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f92620-2422-43e5-95ee-4a71ba7a69d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:47.586437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f92620-2422-43e5-95ee-4a71ba7a69d1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.586718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f92620-2422-43e5-95ee-4a71ba7a69d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.586997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f92620-2422-43e5-95ee-4a71ba7a69d1 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.587841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f92620-2422-43e5-95ee-4a71ba7a69d1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:47.590416 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f92620-2422-43e5-95ee-4a71ba7a69d1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:47.608570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 381/771] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/services => generated 772 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:47.610548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae6969-1c77-4fee-bee1-a9fff4a85f64 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.610893 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae6969-1c77-4fee-bee1-a9fff4a85f64 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.611273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beae6969-1c77-4fee-bee1-a9fff4a85f64 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.613153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 382/772] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:47.613965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae6969-1c77-4fee-bee1-a9fff4a85f64 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:47.651463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 391/773] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:47.654040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779ce26-021a-42e9-bc9d-e17646c374ff None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.654474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779ce26-021a-42e9-bc9d-e17646c374ff None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.654876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2779ce26-021a-42e9-bc9d-e17646c374ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.676344 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2779ce26-021a-42e9-bc9d-e17646c374ff None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:47.677861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2779ce26-021a-42e9-bc9d-e17646c374ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:47.763129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beae6969-1c77-4fee-bee1-a9fff4a85f64 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:47.778563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 392/774] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:47.779519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779ce26-021a-42e9-bc9d-e17646c374ff None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:47.809203 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c1f72-4938-4b18-8558-88edd44ed026 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:47.810960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c1f72-4938-4b18-8558-88edd44ed026 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.811143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c1f72-4938-4b18-8558-88edd44ed026 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.811425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1c1f72-4938-4b18-8558-88edd44ed026 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.820256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c1f72-4938-4b18-8558-88edd44ed026 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:47.821741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1c1f72-4938-4b18-8558-88edd44ed026 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-de1c1f72-4938-4b18-8558-88edd44ed026 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:47.838150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:47.839925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:47.840905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:47.840905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:47.840905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:47.840905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:47.840905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:47.840905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:47.840905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:47.840905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:47.840905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 383/775] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:47.841406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c1f72-4938-4b18-8558-88edd44ed026 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:47.888287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:47.911976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2779ce26-021a-42e9-bc9d-e17646c374ff None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:47.913151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:47.947689 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cfbcd-5ba9-418d-a73b-6bbc396e6eba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:47.948946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cfbcd-5ba9-418d-a73b-6bbc396e6eba None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:47.949233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cfbcd-5ba9-418d-a73b-6bbc396e6eba None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:47.949525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a2cfbcd-5ba9-418d-a73b-6bbc396e6eba None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:47.950322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cfbcd-5ba9-418d-a73b-6bbc396e6eba None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:47.951533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a2cfbcd-5ba9-418d-a73b-6bbc396e6eba None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-7a2cfbcd-5ba9-418d-a73b-6bbc396e6eba None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:47.963735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:47.965560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:10:47.967186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:47.971491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 393/776] 10.222.0.69 () {56 vars in 1129 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:47.972338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cfbcd-5ba9-418d-a73b-6bbc396e6eba None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:47.975474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1c1f72-4938-4b18-8558-88edd44ed026 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.014983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d658c-ce4a-4066-8c3e-ec15f473835e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.016130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d658c-ce4a-4066-8c3e-ec15f473835e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.016565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d658c-ce4a-4066-8c3e-ec15f473835e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.016990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8d658c-ce4a-4066-8c3e-ec15f473835e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.029064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8d658c-ce4a-4066-8c3e-ec15f473835e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-bc8d658c-ce4a-4066-8c3e-ec15f473835e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:48.037782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:48.038924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:48.039851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:48.042872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 384/777] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:48.043948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d658c-ce4a-4066-8c3e-ec15f473835e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.084164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a2cfbcd-5ba9-418d-a73b-6bbc396e6eba None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.119119 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47573e07-4322-4a66-943d-3ddf5ab009bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.120113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47573e07-4322-4a66-943d-3ddf5ab009bd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.120113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47573e07-4322-4a66-943d-3ddf5ab009bd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.120441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47573e07-4322-4a66-943d-3ddf5ab009bd None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.121945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47573e07-4322-4a66-943d-3ddf5ab009bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:48.122304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47573e07-4322-4a66-943d-3ddf5ab009bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.123865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47573e07-4322-4a66-943d-3ddf5ab009bd None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.137655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 394/778] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:48.138711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47573e07-4322-4a66-943d-3ddf5ab009bd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.141596 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.162993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8d658c-ce4a-4066-8c3e-ec15f473835e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.175246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.203713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ca370-07cf-407c-9801-00bf87f46d30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.204564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ca370-07cf-407c-9801-00bf87f46d30 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.204954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ca370-07cf-407c-9801-00bf87f46d30 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.205333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5ca370-07cf-407c-9801-00bf87f46d30 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.206177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ca370-07cf-407c-9801-00bf87f46d30 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:48.206597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ca370-07cf-407c-9801-00bf87f46d30 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.208190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5ca370-07cf-407c-9801-00bf87f46d30 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.213710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 385/779] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:48.215539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ca370-07cf-407c-9801-00bf87f46d30 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.261848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47573e07-4322-4a66-943d-3ddf5ab009bd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.302915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35d84d-85fc-4cab-b855-f87946b05776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.303829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d84d-85fc-4cab-b855-f87946b05776 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.304283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d84d-85fc-4cab-b855-f87946b05776 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.304701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d35d84d-85fc-4cab-b855-f87946b05776 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.305739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35d84d-85fc-4cab-b855-f87946b05776 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:48.306291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35d84d-85fc-4cab-b855-f87946b05776 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.308394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d35d84d-85fc-4cab-b855-f87946b05776 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.322253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 395/780] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:48.323517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35d84d-85fc-4cab-b855-f87946b05776 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.355660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5ca370-07cf-407c-9801-00bf87f46d30 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.391570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f23a8b-066f-4ac8-a9a0-138a60e9499e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.392717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f23a8b-066f-4ac8-a9a0-138a60e9499e None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.392996 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f23a8b-066f-4ac8-a9a0-138a60e9499e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.393275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f23a8b-066f-4ac8-a9a0-138a60e9499e None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/36955d7d4505457696ab6177adb999a9/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.394809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.395300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f23a8b-066f-4ac8-a9a0-138a60e9499e None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=36955d7d4505457696ab6177adb999a9, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.396790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f23a8b-066f-4ac8-a9a0-138a60e9499e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.410922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 386/781] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:48 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/36955d7d4505457696ab6177adb999a9/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:48.412279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69d7a1-c30f-4dac-8408-cf88bf51f56e None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.412568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69d7a1-c30f-4dac-8408-cf88bf51f56e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.412843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d69d7a1-c30f-4dac-8408-cf88bf51f56e None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.414311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 387/782] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:48.414929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69d7a1-c30f-4dac-8408-cf88bf51f56e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.457821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.462641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d35d84d-85fc-4cab-b855-f87946b05776 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.518695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a6a876-f939-4d6f-ba1b-d3c47264b15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.519378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6a876-f939-4d6f-ba1b-d3c47264b15a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.519665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6a876-f939-4d6f-ba1b-d3c47264b15a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.519944 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a6a876-f939-4d6f-ba1b-d3c47264b15a None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.520661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6a876-f939-4d6f-ba1b-d3c47264b15a None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:48.520971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6a876-f939-4d6f-ba1b-d3c47264b15a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.522384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a6a876-f939-4d6f-ba1b-d3c47264b15a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.529024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 396/783] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:48.530463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003369a-9328-4f36-9577-6443697c4130 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.530754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003369a-9328-4f36-9577-6443697c4130 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.531031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e003369a-9328-4f36-9577-6443697c4130 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.550377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d69d7a1-c30f-4dac-8408-cf88bf51f56e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.550800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e003369a-9328-4f36-9577-6443697c4130 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:48.552030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e003369a-9328-4f36-9577-6443697c4130 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:48.595051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45046a70-6f37-4813-a2cc-96eabd8fbcac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.596323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45046a70-6f37-4813-a2cc-96eabd8fbcac None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.596828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45046a70-6f37-4813-a2cc-96eabd8fbcac None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.597318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45046a70-6f37-4813-a2cc-96eabd8fbcac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.610760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45046a70-6f37-4813-a2cc-96eabd8fbcac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.612342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45046a70-6f37-4813-a2cc-96eabd8fbcac None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.628501 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 397/784] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:48 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:48.629422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003369a-9328-4f36-9577-6443697c4130 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-45046a70-6f37-4813-a2cc-96eabd8fbcac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:48.630990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:48.631851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:48.632733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:48.632733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:48.632733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:48.632733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:48.632733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 388/785] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:48.639316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45046a70-6f37-4813-a2cc-96eabd8fbcac None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.701609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.746506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e003369a-9328-4f36-9577-6443697c4130 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.752944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.771474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45046a70-6f37-4813-a2cc-96eabd8fbcac None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.783959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e92bf-c7a4-475e-9a1a-fc7c72ce2a68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.784691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e92bf-c7a4-475e-9a1a-fc7c72ce2a68 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.784979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e92bf-c7a4-475e-9a1a-fc7c72ce2a68 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.785259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c0e92bf-c7a4-475e-9a1a-fc7c72ce2a68 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.786124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e92bf-c7a4-475e-9a1a-fc7c72ce2a68 None admin] RBAC: Adding query filter params (interface=public, service_id=3fbf60cbc94448cfad5481ee2a11ebda, region_id=RegionOne) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:48.786457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e92bf-c7a4-475e-9a1a-fc7c72ce2a68 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.787815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c0e92bf-c7a4-475e-9a1a-fc7c72ce2a68 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.797156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 398/786] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/endpoints?service_id=3fbf60cbc94448cfad5481ee2a11ebda&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:48.798154 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e92bf-c7a4-475e-9a1a-fc7c72ce2a68 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.812446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503179a-605d-4049-9d10-7e4ad88d9af7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.813470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503179a-605d-4049-9d10-7e4ad88d9af7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.814061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503179a-605d-4049-9d10-7e4ad88d9af7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.814561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d503179a-605d-4049-9d10-7e4ad88d9af7 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.815507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503179a-605d-4049-9d10-7e4ad88d9af7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:48.816016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503179a-605d-4049-9d10-7e4ad88d9af7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:48.817845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d503179a-605d-4049-9d10-7e4ad88d9af7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:48.832921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 389/787] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:48.834322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503179a-605d-4049-9d10-7e4ad88d9af7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.910902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c0e92bf-c7a4-475e-9a1a-fc7c72ce2a68 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.934811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:48.947070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70401c6e-2d75-4268-8ac7-0e645ab8f5b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:48.947863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70401c6e-2d75-4268-8ac7-0e645ab8f5b0 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:48.948152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70401c6e-2d75-4268-8ac7-0e645ab8f5b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:48.948525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70401c6e-2d75-4268-8ac7-0e645ab8f5b0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:48.953003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d503179a-605d-4049-9d10-7e4ad88d9af7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:48.961126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70401c6e-2d75-4268-8ac7-0e645ab8f5b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-70401c6e-2d75-4268-8ac7-0e645ab8f5b0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:48.969726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:48.970929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:48.971958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:48.975901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 399/788] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:48.976748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70401c6e-2d75-4268-8ac7-0e645ab8f5b0 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:48.994124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b3351-6cfe-4326-830f-b57cd0ac1154 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.003410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b3351-6cfe-4326-830f-b57cd0ac1154 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.003410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b3351-6cfe-4326-830f-b57cd0ac1154 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.003410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4b3351-6cfe-4326-830f-b57cd0ac1154 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.003410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b3351-6cfe-4326-830f-b57cd0ac1154 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.003410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4b3351-6cfe-4326-830f-b57cd0ac1154 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.013630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 390/789] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:49.014556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b3351-6cfe-4326-830f-b57cd0ac1154 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.044272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.121943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70401c6e-2d75-4268-8ac7-0e645ab8f5b0 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.208936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4dd31-e876-4d56-b546-d65b0c554230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.210258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4dd31-e876-4d56-b546-d65b0c554230 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.210851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4dd31-e876-4d56-b546-d65b0c554230 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.211334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4dd31-e876-4d56-b546-d65b0c554230 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.217146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4b3351-6cfe-4326-830f-b57cd0ac1154 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.221957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4dd31-e876-4d56-b546-d65b0c554230 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.223904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4dd31-e876-4d56-b546-d65b0c554230 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f2c4dd31-e876-4d56-b546-d65b0c554230 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:49.248960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:49.250132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:49.251085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:49.251085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:49.251085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:49.251085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:49.251085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:49.251085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:49.251085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:49.251085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:49.251085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:10:49.251085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:49.252365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 400/790] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Jul 20 19:10:48 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:49.257142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4dd31-e876-4d56-b546-d65b0c554230 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.262761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4a6b7-2f2c-4ec4-b5cf-24c1ab5eadfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.263783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a6b7-2f2c-4ec4-b5cf-24c1ab5eadfd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.264263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a6b7-2f2c-4ec4-b5cf-24c1ab5eadfd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.264741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cf4a6b7-2f2c-4ec4-b5cf-24c1ab5eadfd None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.265767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4a6b7-2f2c-4ec4-b5cf-24c1ab5eadfd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:49.266274 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4a6b7-2f2c-4ec4-b5cf-24c1ab5eadfd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.268410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cf4a6b7-2f2c-4ec4-b5cf-24c1ab5eadfd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.280747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 391/791] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:49.285903 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4a6b7-2f2c-4ec4-b5cf-24c1ab5eadfd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.313411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.385939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.403998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4dd31-e876-4d56-b546-d65b0c554230 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.421916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cf4a6b7-2f2c-4ec4-b5cf-24c1ab5eadfd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.459391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789aabf-ecea-4bd2-8312-5e2428a3afa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.460658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789aabf-ecea-4bd2-8312-5e2428a3afa9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.461195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789aabf-ecea-4bd2-8312-5e2428a3afa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.461728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d789aabf-ecea-4bd2-8312-5e2428a3afa9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.473960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789aabf-ecea-4bd2-8312-5e2428a3afa9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.475838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d789aabf-ecea-4bd2-8312-5e2428a3afa9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-d789aabf-ecea-4bd2-8312-5e2428a3afa9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:49.496475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:49.497436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:49.498362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:49.498362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:49.498362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:49.498362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:49.500640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 392/792] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:49.507008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117141b-9aca-4220-9ba6-5729486262e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.507698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117141b-9aca-4220-9ba6-5729486262e9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.507984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117141b-9aca-4220-9ba6-5729486262e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.508265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6117141b-9aca-4220-9ba6-5729486262e9 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.508510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789aabf-ecea-4bd2-8312-5e2428a3afa9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.512349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117141b-9aca-4220-9ba6-5729486262e9 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:49.512349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117141b-9aca-4220-9ba6-5729486262e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.512349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6117141b-9aca-4220-9ba6-5729486262e9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.530206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 401/793] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:49.540749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117141b-9aca-4220-9ba6-5729486262e9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.625013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d789aabf-ecea-4bd2-8312-5e2428a3afa9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.655874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.685617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f07e4a-1363-40d6-8631-120050378c7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.686311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f07e4a-1363-40d6-8631-120050378c7a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.686597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f07e4a-1363-40d6-8631-120050378c7a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.686877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f07e4a-1363-40d6-8631-120050378c7a None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.687622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f07e4a-1363-40d6-8631-120050378c7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:49.687932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f07e4a-1363-40d6-8631-120050378c7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.689365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f07e4a-1363-40d6-8631-120050378c7a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.709730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 393/794] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:49.715784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:49.716713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f07e4a-1363-40d6-8631-120050378c7a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.721872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6117141b-9aca-4220-9ba6-5729486262e9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.783441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc96519-2b1e-48ab-8eb7-5d40f9aaf43a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.784239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc96519-2b1e-48ab-8eb7-5d40f9aaf43a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.784526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc96519-2b1e-48ab-8eb7-5d40f9aaf43a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.784805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc96519-2b1e-48ab-8eb7-5d40f9aaf43a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.803185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc96519-2b1e-48ab-8eb7-5d40f9aaf43a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.804643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc96519-2b1e-48ab-8eb7-5d40f9aaf43a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-4dc96519-2b1e-48ab-8eb7-5d40f9aaf43a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:49.833798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:49.834754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:49.836101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:49.836101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:49.836101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:49.836101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:49.836101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 402/795] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:49.842579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc96519-2b1e-48ab-8eb7-5d40f9aaf43a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:49.886340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f07e4a-1363-40d6-8631-120050378c7a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:49.925391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc301de-e7e4-407b-8483-02d7c5385cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:49.926343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc301de-e7e4-407b-8483-02d7c5385cab None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:49.926631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc301de-e7e4-407b-8483-02d7c5385cab None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:49.926909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc301de-e7e4-407b-8483-02d7c5385cab None admin] PATH_INFO: `/v3/users/swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:49.935472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc301de-e7e4-407b-8483-02d7c5385cab None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:49.937195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc301de-e7e4-407b-8483-02d7c5385cab None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-adc301de-e7e4-407b-8483-02d7c5385cab None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:49.954625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:49.955623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:49.957018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 394/796] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/users/swift => generated 83 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:49.961622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc301de-e7e4-407b-8483-02d7c5385cab None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.033343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.046086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc96519-2b1e-48ab-8eb7-5d40f9aaf43a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.081680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f82142a-df77-4333-8d6b-a019313a7238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.082345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82142a-df77-4333-8d6b-a019313a7238 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.082629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82142a-df77-4333-8d6b-a019313a7238 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.082904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f82142a-df77-4333-8d6b-a019313a7238 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.083722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82142a-df77-4333-8d6b-a019313a7238 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.084031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82142a-df77-4333-8d6b-a019313a7238 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.085494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f82142a-df77-4333-8d6b-a019313a7238 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.093941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc301de-e7e4-407b-8483-02d7c5385cab None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.099082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 403/797] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.100364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce424a-eb7a-4f7b-bf17-807258a4b0ea None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.100648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce424a-eb7a-4f7b-bf17-807258a4b0ea None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.100925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ce424a-eb7a-4f7b-bf17-807258a4b0ea None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.102416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 404/798] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:50.108994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce424a-eb7a-4f7b-bf17-807258a4b0ea None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.127872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.139686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ea0436-8caf-48ca-b9bb-1c16967d3457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.140665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea0436-8caf-48ca-b9bb-1c16967d3457 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.141172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea0436-8caf-48ca-b9bb-1c16967d3457 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.141765 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ea0436-8caf-48ca-b9bb-1c16967d3457 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.142662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea0436-8caf-48ca-b9bb-1c16967d3457 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.142973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea0436-8caf-48ca-b9bb-1c16967d3457 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.144565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ea0436-8caf-48ca-b9bb-1c16967d3457 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.167814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 395/799] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Jul 20 19:10:49 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.169890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c66ca8-6d5d-450b-85f9-d70a350961fa None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.170374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c66ca8-6d5d-450b-85f9-d70a350961fa None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.170833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c66ca8-6d5d-450b-85f9-d70a350961fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.188375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-34c66ca8-6d5d-450b-85f9-d70a350961fa None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:50.189890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-34c66ca8-6d5d-450b-85f9-d70a350961fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:50.230314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ce424a-eb7a-4f7b-bf17-807258a4b0ea None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.281754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2987fa7-818e-472b-8463-9e361fa5fd4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.281754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2987fa7-818e-472b-8463-9e361fa5fd4d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.281754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2987fa7-818e-472b-8463-9e361fa5fd4d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.281754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2987fa7-818e-472b-8463-9e361fa5fd4d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.282314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 396/800] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:50.283511 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2987fa7-818e-472b-8463-9e361fa5fd4d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.284033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c66ca8-6d5d-450b-85f9-d70a350961fa None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.284964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2987fa7-818e-472b-8463-9e361fa5fd4d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f2987fa7-818e-472b-8463-9e361fa5fd4d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:50.302616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:50.303570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:50.304497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:50.304497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:50.304497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:50.304497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:50.304497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:50.304497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:50.304497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:50.304497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:50.309402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 405/801] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:50.310683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2987fa7-818e-472b-8463-9e361fa5fd4d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.349295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.412653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.441952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c66ca8-6d5d-450b-85f9-d70a350961fa None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.454441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2987fa7-818e-472b-8463-9e361fa5fd4d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.480973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de9f2b-7ed6-4915-8d05-7f14b3907400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.482073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de9f2b-7ed6-4915-8d05-7f14b3907400 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.482614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de9f2b-7ed6-4915-8d05-7f14b3907400 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.483040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88de9f2b-7ed6-4915-8d05-7f14b3907400 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.495072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de9f2b-7ed6-4915-8d05-7f14b3907400 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.496377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21fbff-22af-48a2-a23a-b6798644df7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.497254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21fbff-22af-48a2-a23a-b6798644df7d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.497638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21fbff-22af-48a2-a23a-b6798644df7d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.497995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f21fbff-22af-48a2-a23a-b6798644df7d None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.498503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88de9f2b-7ed6-4915-8d05-7f14b3907400 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.498936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21fbff-22af-48a2-a23a-b6798644df7d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.499327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21fbff-22af-48a2-a23a-b6798644df7d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.500989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f21fbff-22af-48a2-a23a-b6798644df7d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.516817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 406/802] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.518836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21fbff-22af-48a2-a23a-b6798644df7d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-88de9f2b-7ed6-4915-8d05-7f14b3907400 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:50.520524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:50.521579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:50.522499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:50.522499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:50.522499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:50.522499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:50.524427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 397/803] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:50.525337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de9f2b-7ed6-4915-8d05-7f14b3907400 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.627779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.645000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f21fbff-22af-48a2-a23a-b6798644df7d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.649543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88de9f2b-7ed6-4915-8d05-7f14b3907400 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.677349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.685153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b3a79-a0c1-4cd2-ba56-6d017ff5af92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.686283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b3a79-a0c1-4cd2-ba56-6d017ff5af92 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.686747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b3a79-a0c1-4cd2-ba56-6d017ff5af92 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.687211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b3a79-a0c1-4cd2-ba56-6d017ff5af92 None admin] PATH_INFO: `/v3/services/network` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.688048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b3a79-a0c1-4cd2-ba56-6d017ff5af92 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.689625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b3a79-a0c1-4cd2-ba56-6d017ff5af92 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.690807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1922df-e300-4ee0-8fb0-330d9e416dec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.691795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1922df-e300-4ee0-8fb0-330d9e416dec None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.692281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1922df-e300-4ee0-8fb0-330d9e416dec None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.692634 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b1922df-e300-4ee0-8fb0-330d9e416dec None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/cbd4016436b54a0fb8c74c35273957a0/roles/5b019d81d9cf4a16b6c6d1caafec9081` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-ce9b3a79-a0c1-4cd2-ba56-6d017ff5af92 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:50.704075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:50.705095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 20 19:10:50.705994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:50.708684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 407/804] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/services/network => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:50.709826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b3a79-a0c1-4cd2-ba56-6d017ff5af92 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.715297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1922df-e300-4ee0-8fb0-330d9e416dec None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=cbd4016436b54a0fb8c74c35273957a0, role_id=5b019d81d9cf4a16b6c6d1caafec9081)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.717716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b1922df-e300-4ee0-8fb0-330d9e416dec None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.740098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 398/805] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:50 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/cbd4016436b54a0fb8c74c35273957a0/roles/5b019d81d9cf4a16b6c6d1caafec9081 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:50.741014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1922df-e300-4ee0-8fb0-330d9e416dec None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.866631 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b3a79-a0c1-4cd2-ba56-6d017ff5af92 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.875675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b1922df-e300-4ee0-8fb0-330d9e416dec None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:50.916911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dfa051-e727-4f44-8559-d12698a3f225 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.917692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa051-e727-4f44-8559-d12698a3f225 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.917808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c91821ca-3f7a-4178-b838-c8961290a0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:50.918049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa051-e727-4f44-8559-d12698a3f225 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.918303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dfa051-e727-4f44-8559-d12698a3f225 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.918461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91821ca-3f7a-4178-b838-c8961290a0fd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.918711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91821ca-3f7a-4178-b838-c8961290a0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.919026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c91821ca-3f7a-4178-b838-c8961290a0fd None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.919122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfa051-e727-4f44-8559-d12698a3f225 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.919337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfa051-e727-4f44-8559-d12698a3f225 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.919704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91821ca-3f7a-4178-b838-c8961290a0fd None admin] RBAC: Adding query filter params (name=network) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:50.920016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91821ca-3f7a-4178-b838-c8961290a0fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:50.920716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dfa051-e727-4f44-8559-d12698a3f225 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.921417 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c91821ca-3f7a-4178-b838-c8961290a0fd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:50.922691 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:50.927900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 399/806] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/services?name=network => generated 124 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.933973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d6a62-cbf0-4ca7-bc2f-76788f7ae260 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.934124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d6a62-cbf0-4ca7-bc2f-76788f7ae260 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.934313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1d6a62-cbf0-4ca7-bc2f-76788f7ae260 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.935850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 400/807] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:50 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:50.936356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d6a62-cbf0-4ca7-bc2f-76788f7ae260 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:50.938573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 408/808] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:50.941400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fadd13-1b3f-4b51-bb82-d84359359fa8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:50.941744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fadd13-1b3f-4b51-bb82-d84359359fa8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:50.942029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fadd13-1b3f-4b51-bb82-d84359359fa8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:50.961086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c5fadd13-1b3f-4b51-bb82-d84359359fa8 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:50.962329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c5fadd13-1b3f-4b51-bb82-d84359359fa8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:51.019963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.040242 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 409/809] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:10:51.041110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fadd13-1b3f-4b51-bb82-d84359359fa8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.084126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1d6a62-cbf0-4ca7-bc2f-76788f7ae260 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.124081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adbc2f-4199-488e-bd44-ea6e9c806ab3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.124819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adbc2f-4199-488e-bd44-ea6e9c806ab3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.125103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adbc2f-4199-488e-bd44-ea6e9c806ab3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.125412 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7adbc2f-4199-488e-bd44-ea6e9c806ab3 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.126504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adbc2f-4199-488e-bd44-ea6e9c806ab3 None admin] RBAC: Adding query filter params (type=network) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:51.126504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adbc2f-4199-488e-bd44-ea6e9c806ab3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.127931 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7adbc2f-4199-488e-bd44-ea6e9c806ab3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.134250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 401/810] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:10:50 2022] GET /identity/v3/services?type=network => generated 350 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:51.135139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adbc2f-4199-488e-bd44-ea6e9c806ab3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.172335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5fadd13-1b3f-4b51-bb82-d84359359fa8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.205118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.220451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c8393-72ed-4506-bc7c-7197dd0e8efc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.222123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c8393-72ed-4506-bc7c-7197dd0e8efc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.222612 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c8393-72ed-4506-bc7c-7197dd0e8efc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.223093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d16c8393-72ed-4506-bc7c-7197dd0e8efc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.233926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c8393-72ed-4506-bc7c-7197dd0e8efc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.235713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d16c8393-72ed-4506-bc7c-7197dd0e8efc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.250125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7adbc2f-4199-488e-bd44-ea6e9c806ab3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-d16c8393-72ed-4506-bc7c-7197dd0e8efc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:51.256224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:51.257127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:51.258063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:51.258063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:51.258063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:51.258063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:51.258063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:51.258063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:51.258063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:51.258063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:51.261690 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 410/811] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:51.263290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c8393-72ed-4506-bc7c-7197dd0e8efc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.300714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60528a-aaa6-435c-b3d8-2242af431d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.301746 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60528a-aaa6-435c-b3d8-2242af431d27 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.302038 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60528a-aaa6-435c-b3d8-2242af431d27 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.302322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb60528a-aaa6-435c-b3d8-2242af431d27 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.315895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb60528a-aaa6-435c-b3d8-2242af431d27 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-eb60528a-aaa6-435c-b3d8-2242af431d27 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:51.325358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:51.326503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:51.327720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 402/812] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:51.328853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60528a-aaa6-435c-b3d8-2242af431d27 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.333529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.392258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d16c8393-72ed-4506-bc7c-7197dd0e8efc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.428381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ddf39-0b18-4cff-9fe5-25f2f439db9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.429161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ddf39-0b18-4cff-9fe5-25f2f439db9f None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.429454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ddf39-0b18-4cff-9fe5-25f2f439db9f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.429762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-647ddf39-0b18-4cff-9fe5-25f2f439db9f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.430500 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ddf39-0b18-4cff-9fe5-25f2f439db9f None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.431929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-647ddf39-0b18-4cff-9fe5-25f2f439db9f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.445454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb60528a-aaa6-435c-b3d8-2242af431d27 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.458180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 411/813] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Jul 20 19:10:51 2022] POST /identity/v3/endpoints => generated 327 bytes in 196 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:51.459013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ddf39-0b18-4cff-9fe5-25f2f439db9f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.496410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.518410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8664bc9a-696b-455f-be49-184571a74cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.519901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664bc9a-696b-455f-be49-184571a74cc5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.520261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664bc9a-696b-455f-be49-184571a74cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.520925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8664bc9a-696b-455f-be49-184571a74cc5 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.521853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664bc9a-696b-455f-be49-184571a74cc5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:51.522497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664bc9a-696b-455f-be49-184571a74cc5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.524549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8664bc9a-696b-455f-be49-184571a74cc5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.551923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 403/814] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:51.554492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2550d78-06d0-48cf-aaa6-533672ea951a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.554787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2550d78-06d0-48cf-aaa6-533672ea951a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.555073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2550d78-06d0-48cf-aaa6-533672ea951a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.556807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 404/815] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:51.557592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2550d78-06d0-48cf-aaa6-533672ea951a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.592310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-647ddf39-0b18-4cff-9fe5-25f2f439db9f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.594488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.641577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e38f9f3-e064-4b84-bd67-03af41f3a558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.642358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38f9f3-e064-4b84-bd67-03af41f3a558 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.642645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38f9f3-e064-4b84-bd67-03af41f3a558 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.642931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e38f9f3-e064-4b84-bd67-03af41f3a558 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.643688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e38f9f3-e064-4b84-bd67-03af41f3a558 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:51.644002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e38f9f3-e064-4b84-bd67-03af41f3a558 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.646303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e38f9f3-e064-4b84-bd67-03af41f3a558 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.657071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 412/816] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:51.658453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7299e-e9a8-4ac7-b387-17e600450ee5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.658754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7299e-e9a8-4ac7-b387-17e600450ee5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.659091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a7299e-e9a8-4ac7-b387-17e600450ee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.679124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-63a7299e-e9a8-4ac7-b387-17e600450ee5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:51.680327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-63a7299e-e9a8-4ac7-b387-17e600450ee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:51.706640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2550d78-06d0-48cf-aaa6-533672ea951a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.746389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf2aeb-f4d9-4a84-ac3e-61ab68d630e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.747254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2aeb-f4d9-4a84-ac3e-61ab68d630e3 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.747539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2aeb-f4d9-4a84-ac3e-61ab68d630e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.747846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2aeb-f4d9-4a84-ac3e-61ab68d630e3 None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/9d8387328e4145db80782f2d54c0966d/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.749911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf2aeb-f4d9-4a84-ac3e-61ab68d630e3 None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=9d8387328e4145db80782f2d54c0966d, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.751374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf2aeb-f4d9-4a84-ac3e-61ab68d630e3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.766039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 405/817] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:10:51 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/9d8387328e4145db80782f2d54c0966d/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:10:51.766547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 413/818] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:51 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:51.767380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf2aeb-f4d9-4a84-ac3e-61ab68d630e3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.793482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7299e-e9a8-4ac7-b387-17e600450ee5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:51.826041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.894362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf2aeb-f4d9-4a84-ac3e-61ab68d630e3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.939422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829bedd-787e-46e2-a469-17515845d4a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:51.940586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829bedd-787e-46e2-a469-17515845d4a6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:51.940905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829bedd-787e-46e2-a469-17515845d4a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:51.941153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829bedd-787e-46e2-a469-17515845d4a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:51.945657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:51.954345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829bedd-787e-46e2-a469-17515845d4a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:51.955400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829bedd-787e-46e2-a469-17515845d4a6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:51.958343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a7299e-e9a8-4ac7-b387-17e600450ee5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8829bedd-787e-46e2-a469-17515845d4a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:51.980571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:51.981752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:51.982681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:51.982681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:51.982681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:51.982681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:51.982681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 406/819] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:51.987782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829bedd-787e-46e2-a469-17515845d4a6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.004481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9febae9-e9a9-4f81-81a6-1f59cb278629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.006066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9febae9-e9a9-4f81-81a6-1f59cb278629 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.006763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9febae9-e9a9-4f81-81a6-1f59cb278629 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.007192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9febae9-e9a9-4f81-81a6-1f59cb278629 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.008251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9febae9-e9a9-4f81-81a6-1f59cb278629 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.010183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9febae9-e9a9-4f81-81a6-1f59cb278629 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-c9febae9-e9a9-4f81-81a6-1f59cb278629 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:52.025490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:52.027497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:52.028575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 414/820] 10.222.0.69 () {56 vars in 1129 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:52.033054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9febae9-e9a9-4f81-81a6-1f59cb278629 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.107442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.121748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829bedd-787e-46e2-a469-17515845d4a6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.147360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9febae9-e9a9-4f81-81a6-1f59cb278629 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.169395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e63cd9-0ca3-43c3-819e-2c5ceac20aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.170364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e63cd9-0ca3-43c3-819e-2c5ceac20aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.170807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e63cd9-0ca3-43c3-819e-2c5ceac20aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.171729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e63cd9-0ca3-43c3-819e-2c5ceac20aa1 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.172888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e63cd9-0ca3-43c3-819e-2c5ceac20aa1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:52.173349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e63cd9-0ca3-43c3-819e-2c5ceac20aa1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.175855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e63cd9-0ca3-43c3-819e-2c5ceac20aa1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.186129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b9b74-f757-4427-9364-0767264152ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.187092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9b74-f757-4427-9364-0767264152ad None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.187214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9b74-f757-4427-9364-0767264152ad None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.187454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8b9b74-f757-4427-9364-0767264152ad None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.188167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b9b74-f757-4427-9364-0767264152ad None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:52.188484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b9b74-f757-4427-9364-0767264152ad None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.189876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8b9b74-f757-4427-9364-0767264152ad None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.191520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 407/821] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:52.197542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 415/822] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:52.202438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b9b74-f757-4427-9364-0767264152ad None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.202557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e63cd9-0ca3-43c3-819e-2c5ceac20aa1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.230848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.325357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e63cd9-0ca3-43c3-819e-2c5ceac20aa1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.332862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8b9b74-f757-4427-9364-0767264152ad None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.364757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa668-8bd3-46c5-8dc8-7d9f94c72f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.365905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa668-8bd3-46c5-8dc8-7d9f94c72f74 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.366394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa668-8bd3-46c5-8dc8-7d9f94c72f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.366868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9aa668-8bd3-46c5-8dc8-7d9f94c72f74 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.367772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa668-8bd3-46c5-8dc8-7d9f94c72f74 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:52.368332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa668-8bd3-46c5-8dc8-7d9f94c72f74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.370030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9aa668-8bd3-46c5-8dc8-7d9f94c72f74 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.376059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8795cb-aa82-4cb9-a11b-d560e7df7e86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.376843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8795cb-aa82-4cb9-a11b-d560e7df7e86 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.377123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8795cb-aa82-4cb9-a11b-d560e7df7e86 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.377396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8795cb-aa82-4cb9-a11b-d560e7df7e86 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.378516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 408/823] 10.222.0.69 () {56 vars in 1124 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:52.383626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa668-8bd3-46c5-8dc8-7d9f94c72f74 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.391972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8795cb-aa82-4cb9-a11b-d560e7df7e86 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.393770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8795cb-aa82-4cb9-a11b-d560e7df7e86 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.404436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-4f8795cb-aa82-4cb9-a11b-d560e7df7e86 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:52.412864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:52.414032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:52.415011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:52.415011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:52.415011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:52.415011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:52.415011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:52.415011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:52.415011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:52.415011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:52.415011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:10:52.415011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:52.417941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 416/824] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:52.423446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8795cb-aa82-4cb9-a11b-d560e7df7e86 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.514764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.527316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9aa668-8bd3-46c5-8dc8-7d9f94c72f74 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.536744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8795cb-aa82-4cb9-a11b-d560e7df7e86 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.566512 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0dbbd-9964-4489-9537-c0213f0db4aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.567421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0dbbd-9964-4489-9537-c0213f0db4aa None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.567853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0dbbd-9964-4489-9537-c0213f0db4aa None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.568357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e0dbbd-9964-4489-9537-c0213f0db4aa None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.569234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0dbbd-9964-4489-9537-c0213f0db4aa None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.570806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e0dbbd-9964-4489-9537-c0213f0db4aa None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.576208 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f0c61d-6c19-448b-906d-009d03431ca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.577283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0c61d-6c19-448b-906d-009d03431ca3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.577283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0c61d-6c19-448b-906d-009d03431ca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.577773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f0c61d-6c19-448b-906d-009d03431ca3 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.578757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0c61d-6c19-448b-906d-009d03431ca3 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:52.579158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0c61d-6c19-448b-906d-009d03431ca3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.580709 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f0c61d-6c19-448b-906d-009d03431ca3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.604468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 409/826] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Jul 20 19:10:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:10:52.607948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 417/826] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:52.609182 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0dbbd-9964-4489-9537-c0213f0db4aa None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:52.690878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.735334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e0dbbd-9964-4489-9537-c0213f0db4aa None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:52.783439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935656fe-f7ff-4c21-a956-4fec04d9d513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:52.784363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935656fe-f7ff-4c21-a956-4fec04d9d513 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.784798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935656fe-f7ff-4c21-a956-4fec04d9d513 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.785224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935656fe-f7ff-4c21-a956-4fec04d9d513 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.865127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935656fe-f7ff-4c21-a956-4fec04d9d513 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:52.866872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935656fe-f7ff-4c21-a956-4fec04d9d513 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:52.899470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e047aff-35c9-4a76-8da4-93decea609b8 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.899993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e047aff-35c9-4a76-8da4-93decea609b8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.900419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e047aff-35c9-4a76-8da4-93decea609b8 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.902239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 418/827] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:52.907452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf9ed1-cd21-4e23-9335-18cfdef4af37 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:52.907925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf9ed1-cd21-4e23-9335-18cfdef4af37 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:52.908348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9caf9ed1-cd21-4e23-9335-18cfdef4af37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:52.926046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9caf9ed1-cd21-4e23-9335-18cfdef4af37 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:52.927470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9caf9ed1-cd21-4e23-9335-18cfdef4af37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:52.938736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:52.991676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 410/828] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:10:52.999747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935656fe-f7ff-4c21-a956-4fec04d9d513 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.027786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 419/829] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:52 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:10:53.050458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf9ed1-cd21-4e23-9335-18cfdef4af37 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.113188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935656fe-f7ff-4c21-a956-4fec04d9d513 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.126339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:53.159761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9caf9ed1-cd21-4e23-9335-18cfdef4af37 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.160924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbe238-1743-4874-979e-4def533cf58f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.162225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbe238-1743-4874-979e-4def533cf58f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.162784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbe238-1743-4874-979e-4def533cf58f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.163279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2fbe238-1743-4874-979e-4def533cf58f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.178873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbe238-1743-4874-979e-4def533cf58f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.180687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2fbe238-1743-4874-979e-4def533cf58f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.196377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844696d0-dc7f-41f4-9387-9cb688650678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.197169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844696d0-dc7f-41f4-9387-9cb688650678 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.197447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844696d0-dc7f-41f4-9387-9cb688650678 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.197745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844696d0-dc7f-41f4-9387-9cb688650678 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-c2fbe238-1743-4874-979e-4def533cf58f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:53.204848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:53.205768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:53.206678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:53.206678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:53.206678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:53.206678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:53.207459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 411/830] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:53.212319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbe238-1743-4874-979e-4def533cf58f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.213091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844696d0-dc7f-41f4-9387-9cb688650678 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-844696d0-dc7f-41f4-9387-9cb688650678 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:53.221358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:53.222348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:10:53.223545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:53.226287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 420/831] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:53.232688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844696d0-dc7f-41f4-9387-9cb688650678 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.238953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:53.341971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2fbe238-1743-4874-979e-4def533cf58f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.361139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844696d0-dc7f-41f4-9387-9cb688650678 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.379246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c028fe-3819-4aa1-9434-ad7fc27c3e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.380262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c028fe-3819-4aa1-9434-ad7fc27c3e31 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.380744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c028fe-3819-4aa1-9434-ad7fc27c3e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.381214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c028fe-3819-4aa1-9434-ad7fc27c3e31 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.382153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c028fe-3819-4aa1-9434-ad7fc27c3e31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:53.382675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c028fe-3819-4aa1-9434-ad7fc27c3e31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.384271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c028fe-3819-4aa1-9434-ad7fc27c3e31 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.391417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:53.397746 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 412/832] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:53.403848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c028fe-3819-4aa1-9434-ad7fc27c3e31 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.404854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b458e3-1245-40fe-aea8-1cc4c6d9675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.405539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b458e3-1245-40fe-aea8-1cc4c6d9675a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.405837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b458e3-1245-40fe-aea8-1cc4c6d9675a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.406091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b458e3-1245-40fe-aea8-1cc4c6d9675a None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.406833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b458e3-1245-40fe-aea8-1cc4c6d9675a None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:53.407137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b458e3-1245-40fe-aea8-1cc4c6d9675a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.408491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b458e3-1245-40fe-aea8-1cc4c6d9675a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.424159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 421/833] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:53.425200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b458e3-1245-40fe-aea8-1cc4c6d9675a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.516859 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:53.537783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b458e3-1245-40fe-aea8-1cc4c6d9675a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.547862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c028fe-3819-4aa1-9434-ad7fc27c3e31 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.577941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e057e83-a2c3-4daf-a040-983db471cb11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.578796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e057e83-a2c3-4daf-a040-983db471cb11 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.579198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e057e83-a2c3-4daf-a040-983db471cb11 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.579564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e057e83-a2c3-4daf-a040-983db471cb11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.591263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e057e83-a2c3-4daf-a040-983db471cb11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.592918 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e057e83-a2c3-4daf-a040-983db471cb11 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.593109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639ccf5-9833-4462-97c6-62f8494f4a78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.594258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639ccf5-9833-4462-97c6-62f8494f4a78 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.594722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639ccf5-9833-4462-97c6-62f8494f4a78 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.595316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2639ccf5-9833-4462-97c6-62f8494f4a78 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.604594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639ccf5-9833-4462-97c6-62f8494f4a78 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.606438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2639ccf5-9833-4462-97c6-62f8494f4a78 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-7e057e83-a2c3-4daf-a040-983db471cb11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:53.612156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:53.613281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:53.614455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:53.614455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:53.614455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:53.614455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:53.621019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 422/834] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:53.622144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e057e83-a2c3-4daf-a040-983db471cb11 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-2639ccf5-9833-4462-97c6-62f8494f4a78 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:53.626852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:53.627966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:53.629086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:53.629086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:53.629086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:53.629086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:53.629086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:53.629086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:53.629086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:53.629086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:53.636825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 413/835] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:53.637955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639ccf5-9833-4462-97c6-62f8494f4a78 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.675889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:53.736762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e057e83-a2c3-4daf-a040-983db471cb11 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.771943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2639ccf5-9833-4462-97c6-62f8494f4a78 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.780163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033f2ef-1b1e-46c8-9711-e1ddcb8eff3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.783802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033f2ef-1b1e-46c8-9711-e1ddcb8eff3b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.784353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033f2ef-1b1e-46c8-9711-e1ddcb8eff3b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.785253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c033f2ef-1b1e-46c8-9711-e1ddcb8eff3b None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.786610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033f2ef-1b1e-46c8-9711-e1ddcb8eff3b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:53.787155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033f2ef-1b1e-46c8-9711-e1ddcb8eff3b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.789754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c033f2ef-1b1e-46c8-9711-e1ddcb8eff3b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.808735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 423/836] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:53.816300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033f2ef-1b1e-46c8-9711-e1ddcb8eff3b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.816438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a966f-4f12-40c4-a582-462cf66937fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.816438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a966f-4f12-40c4-a582-462cf66937fe None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.816438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a966f-4f12-40c4-a582-462cf66937fe None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.816438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44a966f-4f12-40c4-a582-462cf66937fe None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.816738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a966f-4f12-40c4-a582-462cf66937fe None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:53.816738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a966f-4f12-40c4-a582-462cf66937fe None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.818293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b44a966f-4f12-40c4-a582-462cf66937fe None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.826214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:53.835649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 414/837] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:53.842276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a966f-4f12-40c4-a582-462cf66937fe None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:53.932829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c033f2ef-1b1e-46c8-9711-e1ddcb8eff3b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.935293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:53.956978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44a966f-4f12-40c4-a582-462cf66937fe None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:53.971516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f276e14e-0667-4571-836b-0338ef9c7ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.972320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276e14e-0667-4571-836b-0338ef9c7ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.972602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276e14e-0667-4571-836b-0338ef9c7ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.972874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f276e14e-0667-4571-836b-0338ef9c7ef2 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.981289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276e14e-0667-4571-836b-0338ef9c7ef2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.982820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f276e14e-0667-4571-836b-0338ef9c7ef2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.994929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a5dd77-fa69-4dab-9ad9-d846ca7f08f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:53.996229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5dd77-fa69-4dab-9ad9-d846ca7f08f2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:53.996510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5dd77-fa69-4dab-9ad9-d846ca7f08f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:53.996782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a5dd77-fa69-4dab-9ad9-d846ca7f08f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:53.997636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5dd77-fa69-4dab-9ad9-d846ca7f08f2 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=36955d7d4505457696ab6177adb999a9) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:53.997952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5dd77-fa69-4dab-9ad9-d846ca7f08f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:53.999383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a5dd77-fa69-4dab-9ad9-d846ca7f08f2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f276e14e-0667-4571-836b-0338ef9c7ef2 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:53.999991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:54.000892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:54.001898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 424/838] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:54.002507 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd3b8e-fddf-4e3f-94ae-c6a68654b870 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.008308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd3b8e-fddf-4e3f-94ae-c6a68654b870 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.008308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55dd3b8e-fddf-4e3f-94ae-c6a68654b870 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.008308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 425/839] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:54.008308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd3b8e-fddf-4e3f-94ae-c6a68654b870 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.013174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 415/840] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:53 2022] GET /identity/v3/role_assignments?user.id=36955d7d4505457696ab6177adb999a9&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:54.014944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4546-1988-4230-bac6-5962f1f3942b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.015454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4546-1988-4230-bac6-5962f1f3942b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.016031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b0f4546-1988-4230-bac6-5962f1f3942b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.039676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7b0f4546-1988-4230-bac6-5962f1f3942b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:54.041354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7b0f4546-1988-4230-bac6-5962f1f3942b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:54.123114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.141812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 416/841] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:54.146350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55dd3b8e-fddf-4e3f-94ae-c6a68654b870 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.162858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f4546-1988-4230-bac6-5962f1f3942b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.190953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba759fb-db90-4391-b5e2-26ebf5801c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.191618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba759fb-db90-4391-b5e2-26ebf5801c92 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.191897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba759fb-db90-4391-b5e2-26ebf5801c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.192170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba759fb-db90-4391-b5e2-26ebf5801c92 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.193008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba759fb-db90-4391-b5e2-26ebf5801c92 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:54.193311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba759fb-db90-4391-b5e2-26ebf5801c92 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:54.194772 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba759fb-db90-4391-b5e2-26ebf5801c92 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:54.214794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 426/842] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:54.221348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba759fb-db90-4391-b5e2-26ebf5801c92 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.249534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.291719 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b0f4546-1988-4230-bac6-5962f1f3942b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.333244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db78e53-49b2-4010-b963-7fcfb221e271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.334393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db78e53-49b2-4010-b963-7fcfb221e271 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.334916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db78e53-49b2-4010-b963-7fcfb221e271 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.335333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db78e53-49b2-4010-b963-7fcfb221e271 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.346932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db78e53-49b2-4010-b963-7fcfb221e271 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:54.347991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba759fb-db90-4391-b5e2-26ebf5801c92 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-5db78e53-49b2-4010-b963-7fcfb221e271 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:54.357836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:54.358946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:10:54.359976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:54.362506 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 417/843] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:54.373793 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db78e53-49b2-4010-b963-7fcfb221e271 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.389867 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d53854-a883-45f5-a2e4-dbc0f7991b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.390723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d53854-a883-45f5-a2e4-dbc0f7991b96 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.391117 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d53854-a883-45f5-a2e4-dbc0f7991b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.391505 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d53854-a883-45f5-a2e4-dbc0f7991b96 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.472435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d53854-a883-45f5-a2e4-dbc0f7991b96 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:54.474411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d53854-a883-45f5-a2e4-dbc0f7991b96 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:54.501948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.559449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.599959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 427/844] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:10:54.601081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d53854-a883-45f5-a2e4-dbc0f7991b96 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.623665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db78e53-49b2-4010-b963-7fcfb221e271 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.668967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce145b-9fa1-40ce-acb8-ad62e92d331d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.669813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce145b-9fa1-40ce-acb8-ad62e92d331d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.670097 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce145b-9fa1-40ce-acb8-ad62e92d331d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.670371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ce145b-9fa1-40ce-acb8-ad62e92d331d None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.671122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce145b-9fa1-40ce-acb8-ad62e92d331d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:54.671428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce145b-9fa1-40ce-acb8-ad62e92d331d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:54.672955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ce145b-9fa1-40ce-acb8-ad62e92d331d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:54.687121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 418/845] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:54.688049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce145b-9fa1-40ce-acb8-ad62e92d331d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.730290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d53854-a883-45f5-a2e4-dbc0f7991b96 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.766158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.788506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbebe09-028e-4f13-a122-9d2530988f95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.790267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbebe09-028e-4f13-a122-9d2530988f95 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.791003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbebe09-028e-4f13-a122-9d2530988f95 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.791494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbebe09-028e-4f13-a122-9d2530988f95 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.799655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ce145b-9fa1-40ce-acb8-ad62e92d331d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.806068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbebe09-028e-4f13-a122-9d2530988f95 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:54.808404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbebe09-028e-4f13-a122-9d2530988f95 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:54.816494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-fbbebe09-028e-4f13-a122-9d2530988f95 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:54.829190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:54.830354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:54.831446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:54.831446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:54.831446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:54.831446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:54.835915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 428/846] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:54.837055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e927c18-8f86-4378-8225-0f19a670dd55 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.837345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e927c18-8f86-4378-8225-0f19a670dd55 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.837652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e927c18-8f86-4378-8225-0f19a670dd55 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.841668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 429/847] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:54.841668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e927c18-8f86-4378-8225-0f19a670dd55 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:54.843784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb31b26-af4c-4d8d-9663-d7b3ab2dd4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:54.844863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb31b26-af4c-4d8d-9663-d7b3ab2dd4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.845250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb31b26-af4c-4d8d-9663-d7b3ab2dd4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.845666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb31b26-af4c-4d8d-9663-d7b3ab2dd4e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.858226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb31b26-af4c-4d8d-9663-d7b3ab2dd4e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:54.859876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb31b26-af4c-4d8d-9663-d7b3ab2dd4e2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-bdb31b26-af4c-4d8d-9663-d7b3ab2dd4e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:54.878216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:54.879359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:54.880790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:54.880790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:54.880790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:54.880790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:54.884318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 419/848] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:54.885340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5d6d1-49fc-4c47-a038-c35e329f1834 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:54.886200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5d6d1-49fc-4c47-a038-c35e329f1834 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:54.886622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed5d6d1-49fc-4c47-a038-c35e329f1834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:54.904280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-eed5d6d1-49fc-4c47-a038-c35e329f1834 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:54.905487 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eed5d6d1-49fc-4c47-a038-c35e329f1834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:54.960547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e927c18-8f86-4378-8225-0f19a670dd55 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:54.976227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 420/849] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:54.978683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5d6d1-49fc-4c47-a038-c35e329f1834 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.004780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f07baf-a788-4006-8e79-bfb42aebe4b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.005442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f07baf-a788-4006-8e79-bfb42aebe4b3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.005759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f07baf-a788-4006-8e79-bfb42aebe4b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.006035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f07baf-a788-4006-8e79-bfb42aebe4b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.007380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f07baf-a788-4006-8e79-bfb42aebe4b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.007380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f07baf-a788-4006-8e79-bfb42aebe4b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.008589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f07baf-a788-4006-8e79-bfb42aebe4b3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.027692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 430/850] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.027692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f07baf-a788-4006-8e79-bfb42aebe4b3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.063526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.092065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.099796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eed5d6d1-49fc-4c47-a038-c35e329f1834 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.137123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9381fe-09a4-45f6-a2d2-3cf1d6a2b210 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.137825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9381fe-09a4-45f6-a2d2-3cf1d6a2b210 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.138226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9381fe-09a4-45f6-a2d2-3cf1d6a2b210 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.138541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9381fe-09a4-45f6-a2d2-3cf1d6a2b210 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.138906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f07baf-a788-4006-8e79-bfb42aebe4b3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.139242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9381fe-09a4-45f6-a2d2-3cf1d6a2b210 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.139552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9381fe-09a4-45f6-a2d2-3cf1d6a2b210 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.140934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9381fe-09a4-45f6-a2d2-3cf1d6a2b210 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.153386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 421/851] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.154046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9381fe-09a4-45f6-a2d2-3cf1d6a2b210 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.179998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66b518-dc01-40bb-9d8e-cb1f42b60537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.180853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66b518-dc01-40bb-9d8e-cb1f42b60537 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.181139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66b518-dc01-40bb-9d8e-cb1f42b60537 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.181414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd66b518-dc01-40bb-9d8e-cb1f42b60537 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.182145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66b518-dc01-40bb-9d8e-cb1f42b60537 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.183590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd66b518-dc01-40bb-9d8e-cb1f42b60537 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-dd66b518-dc01-40bb-9d8e-cb1f42b60537 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:55.196055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:55.197197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:55.199101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 431/852] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:55.199906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66b518-dc01-40bb-9d8e-cb1f42b60537 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.269278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9381fe-09a4-45f6-a2d2-3cf1d6a2b210 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.304593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84cf79-2c2c-45c8-b7b6-25148eaf5c37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.305386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84cf79-2c2c-45c8-b7b6-25148eaf5c37 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.305695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84cf79-2c2c-45c8-b7b6-25148eaf5c37 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.305969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e84cf79-2c2c-45c8-b7b6-25148eaf5c37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.306543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.311728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd66b518-dc01-40bb-9d8e-cb1f42b60537 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.315674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84cf79-2c2c-45c8-b7b6-25148eaf5c37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.315674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e84cf79-2c2c-45c8-b7b6-25148eaf5c37 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-3e84cf79-2c2c-45c8-b7b6-25148eaf5c37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:55.330165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:55.331079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:55.332012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:55.332012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:55.332012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:55.332012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:55.332012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:55.332012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:55.332012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:55.332012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:55.332012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 422/853] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:55.332498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84cf79-2c2c-45c8-b7b6-25148eaf5c37 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.351285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.364634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beae4b-6508-4b11-9132-801c5cbdc62b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.365437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beae4b-6508-4b11-9132-801c5cbdc62b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.365766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beae4b-6508-4b11-9132-801c5cbdc62b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.366046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14beae4b-6508-4b11-9132-801c5cbdc62b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.375541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14beae4b-6508-4b11-9132-801c5cbdc62b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.376985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14beae4b-6508-4b11-9132-801c5cbdc62b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-14beae4b-6508-4b11-9132-801c5cbdc62b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:55.392296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:55.398064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 432/854] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:55.399015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beae4b-6508-4b11-9132-801c5cbdc62b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.451543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e84cf79-2c2c-45c8-b7b6-25148eaf5c37 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.489037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309e0d0-8f8e-41d5-9498-0a8121272d1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.489843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309e0d0-8f8e-41d5-9498-0a8121272d1a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.490129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309e0d0-8f8e-41d5-9498-0a8121272d1a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.490526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4309e0d0-8f8e-41d5-9498-0a8121272d1a None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.491232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309e0d0-8f8e-41d5-9498-0a8121272d1a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.491541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309e0d0-8f8e-41d5-9498-0a8121272d1a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.493028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4309e0d0-8f8e-41d5-9498-0a8121272d1a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.503706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 423/855] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.503706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309e0d0-8f8e-41d5-9498-0a8121272d1a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.503993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14beae4b-6508-4b11-9132-801c5cbdc62b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.538195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0710cb-d379-47f9-b07f-d760ca2e6f11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.538777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0710cb-d379-47f9-b07f-d760ca2e6f11 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.539064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0710cb-d379-47f9-b07f-d760ca2e6f11 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.539347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0710cb-d379-47f9-b07f-d760ca2e6f11 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.540134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0710cb-d379-47f9-b07f-d760ca2e6f11 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.540455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0710cb-d379-47f9-b07f-d760ca2e6f11 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.542589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd0710cb-d379-47f9-b07f-d760ca2e6f11 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.561084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 433/856] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.561084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0710cb-d379-47f9-b07f-d760ca2e6f11 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.577572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.598098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.621121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4309e0d0-8f8e-41d5-9498-0a8121272d1a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.659711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84106feb-4220-4a82-a36c-89b68a064cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.660491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84106feb-4220-4a82-a36c-89b68a064cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.660782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84106feb-4220-4a82-a36c-89b68a064cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.661063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84106feb-4220-4a82-a36c-89b68a064cf5 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.661937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84106feb-4220-4a82-a36c-89b68a064cf5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.662252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84106feb-4220-4a82-a36c-89b68a064cf5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.663695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84106feb-4220-4a82-a36c-89b68a064cf5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.666096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0710cb-d379-47f9-b07f-d760ca2e6f11 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.687524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 424/857] 10.222.0.69 () {56 vars in 1135 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.687524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84106feb-4220-4a82-a36c-89b68a064cf5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.708807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf2ca7-8e0f-4307-a85a-7cd269173aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.709477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2ca7-8e0f-4307-a85a-7cd269173aaf None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.709789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2ca7-8e0f-4307-a85a-7cd269173aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.710068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85bf2ca7-8e0f-4307-a85a-7cd269173aaf None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.710776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf2ca7-8e0f-4307-a85a-7cd269173aaf None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.711082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf2ca7-8e0f-4307-a85a-7cd269173aaf None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.712459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85bf2ca7-8e0f-4307-a85a-7cd269173aaf None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.725550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 434/858] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.725550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf2ca7-8e0f-4307-a85a-7cd269173aaf None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:55.813216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84106feb-4220-4a82-a36c-89b68a064cf5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.850872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:55.855772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523d3c2-7ac9-45c8-88a6-54c0b076e6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.856602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523d3c2-7ac9-45c8-88a6-54c0b076e6ec None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.857061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523d3c2-7ac9-45c8-88a6-54c0b076e6ec None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.857540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1523d3c2-7ac9-45c8-88a6-54c0b076e6ec None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:55.858455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523d3c2-7ac9-45c8-88a6-54c0b076e6ec None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=cbd4016436b54a0fb8c74c35273957a0) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:55.859127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523d3c2-7ac9-45c8-88a6-54c0b076e6ec None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:55.860469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1523d3c2-7ac9-45c8-88a6-54c0b076e6ec None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:55.868170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 425/859] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/role_assignments?user.id=cbd4016436b54a0fb8c74c35273957a0&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:55.882532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85bf2ca7-8e0f-4307-a85a-7cd269173aaf None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:55.927443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130acae9-db08-49ce-b9ec-9b82830a8816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:55.928137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130acae9-db08-49ce-b9ec-9b82830a8816 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:55.928455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130acae9-db08-49ce-b9ec-9b82830a8816 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:55.928723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130acae9-db08-49ce-b9ec-9b82830a8816 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.010190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130acae9-db08-49ce-b9ec-9b82830a8816 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.012308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130acae9-db08-49ce-b9ec-9b82830a8816 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.101878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.124010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 435/860] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:10:56.129869 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523d3c2-7ac9-45c8-88a6-54c0b076e6ec None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.133228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.169312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53406d-59e1-4549-9c39-14806f985d76 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.169643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53406d-59e1-4549-9c39-14806f985d76 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.169926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d53406d-59e1-4549-9c39-14806f985d76 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.171287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 436/861] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:10:56.176326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9b866-1f80-42a7-aeca-149875e57707 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.176611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9b866-1f80-42a7-aeca-149875e57707 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.176887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9b866-1f80-42a7-aeca-149875e57707 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.193862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-87a9b866-1f80-42a7-aeca-149875e57707 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:56.195038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-87a9b866-1f80-42a7-aeca-149875e57707 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:56.255802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1523d3c2-7ac9-45c8-88a6-54c0b076e6ec None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.282474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 437/862] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:10:56.302451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2dc1b-830d-4317-b1ff-449c0651ac03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.303250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2dc1b-830d-4317-b1ff-449c0651ac03 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.303531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2dc1b-830d-4317-b1ff-449c0651ac03 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.303806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a2dc1b-830d-4317-b1ff-449c0651ac03 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.306874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9b866-1f80-42a7-aeca-149875e57707 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.317541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2dc1b-830d-4317-b1ff-449c0651ac03 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.317541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a2dc1b-830d-4317-b1ff-449c0651ac03 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-b1a2dc1b-830d-4317-b1ff-449c0651ac03 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:56.336548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:56.337497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:56.338452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:56.338452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:56.338452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:56.338452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:56.338452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 426/863] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:56.344534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2dc1b-830d-4317-b1ff-449c0651ac03 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.405835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.445725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9b866-1f80-42a7-aeca-149875e57707 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.455786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a2dc1b-830d-4317-b1ff-449c0651ac03 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.464816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.498115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b08f55-f950-4d3b-824e-b426791a02c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.498932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08f55-f950-4d3b-824e-b426791a02c4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.499212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08f55-f950-4d3b-824e-b426791a02c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.499483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2b08f55-f950-4d3b-824e-b426791a02c4 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.500133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b08f55-f950-4d3b-824e-b426791a02c4 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.501495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2b08f55-f950-4d3b-824e-b426791a02c4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.503518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c4713-96d2-407f-bac2-b0250a8ae7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.504181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c4713-96d2-407f-bac2-b0250a8ae7cf None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.504460 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c4713-96d2-407f-bac2-b0250a8ae7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.504731 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6c4713-96d2-407f-bac2-b0250a8ae7cf None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.505448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c4713-96d2-407f-bac2-b0250a8ae7cf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:56.505789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c4713-96d2-407f-bac2-b0250a8ae7cf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.507138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6c4713-96d2-407f-bac2-b0250a8ae7cf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.522164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 427/864] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:56.527892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c4713-96d2-407f-bac2-b0250a8ae7cf None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b2b08f55-f950-4d3b-824e-b426791a02c4 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:56.528046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:56.528934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:56.530212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 438/865] 10.222.0.69 () {56 vars in 1132 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:56.535120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b08f55-f950-4d3b-824e-b426791a02c4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.645881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6c4713-96d2-407f-bac2-b0250a8ae7cf None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.647369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2b08f55-f950-4d3b-824e-b426791a02c4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.662267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.687650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c0d35-4734-463f-8d85-22f050284e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.687866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dabc88-a5aa-4151-9597-a6b77dd15935 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.688734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c0d35-4734-463f-8d85-22f050284e6c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.689121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c0d35-4734-463f-8d85-22f050284e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.689350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dabc88-a5aa-4151-9597-a6b77dd15935 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.689602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85c0d35-4734-463f-8d85-22f050284e6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.689962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dabc88-a5aa-4151-9597-a6b77dd15935 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.690399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2dabc88-a5aa-4151-9597-a6b77dd15935 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.691261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dabc88-a5aa-4151-9597-a6b77dd15935 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:56.691816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dabc88-a5aa-4151-9597-a6b77dd15935 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.693437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2dabc88-a5aa-4151-9597-a6b77dd15935 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.697407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c0d35-4734-463f-8d85-22f050284e6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.699112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a85c0d35-4734-463f-8d85-22f050284e6c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.699286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 439/866] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:56.705643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dabc88-a5aa-4151-9597-a6b77dd15935 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a85c0d35-4734-463f-8d85-22f050284e6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:10:56.714861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:56.715980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:56.717093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:10:56.717093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:10:56.717093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:10:56.717093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:10:56.717093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:10:56.717093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:10:56.717093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:10:56.717093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:56.723497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 428/867] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:56.724465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c0d35-4734-463f-8d85-22f050284e6c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:56.762364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:56.827996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2dabc88-a5aa-4151-9597-a6b77dd15935 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.864556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85c0d35-4734-463f-8d85-22f050284e6c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:56.870294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0eb92-299b-46ec-bb2c-318c805b6404 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.871000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0eb92-299b-46ec-bb2c-318c805b6404 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.871391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0eb92-299b-46ec-bb2c-318c805b6404 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.871772 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a0eb92-299b-46ec-bb2c-318c805b6404 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.872664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0eb92-299b-46ec-bb2c-318c805b6404 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:56.873075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0eb92-299b-46ec-bb2c-318c805b6404 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.875036 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a0eb92-299b-46ec-bb2c-318c805b6404 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.889567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 440/868] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:56.906762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc353629-8344-46b1-b58b-8d1fb9188377 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:56.907631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc353629-8344-46b1-b58b-8d1fb9188377 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:56.908022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc353629-8344-46b1-b58b-8d1fb9188377 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:56.908401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc353629-8344-46b1-b58b-8d1fb9188377 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:56.909313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc353629-8344-46b1-b58b-8d1fb9188377 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:56.909768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc353629-8344-46b1-b58b-8d1fb9188377 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:56.911388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc353629-8344-46b1-b58b-8d1fb9188377 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:56.926470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 429/869] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:56.934704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0eb92-299b-46ec-bb2c-318c805b6404 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:57.054375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a0eb92-299b-46ec-bb2c-318c805b6404 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:57.057044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:57.093624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96223397-09bf-4d2c-b8ac-f69f02473904 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:57.094335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96223397-09bf-4d2c-b8ac-f69f02473904 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.094625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96223397-09bf-4d2c-b8ac-f69f02473904 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.094906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96223397-09bf-4d2c-b8ac-f69f02473904 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.095743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96223397-09bf-4d2c-b8ac-f69f02473904 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=9d8387328e4145db80782f2d54c0966d) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:57.096060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96223397-09bf-4d2c-b8ac-f69f02473904 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:57.097502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96223397-09bf-4d2c-b8ac-f69f02473904 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:57.105286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 441/870] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:10:56 2022] GET /identity/v3/role_assignments?user.id=9d8387328e4145db80782f2d54c0966d&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 608 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:57.974439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e69c46-c352-4cb7-9394-f445b21ee0c7 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.974966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e69c46-c352-4cb7-9394-f445b21ee0c7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.974966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e69c46-c352-4cb7-9394-f445b21ee0c7 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:57.976601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 430/871] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:57.982131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c3930-90ac-4c85-ad0a-4a8d8febf8ce None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:57.982565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c3930-90ac-4c85-ad0a-4a8d8febf8ce None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:57.982771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4c3930-90ac-4c85-ad0a-4a8d8febf8ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.000487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ee4c3930-90ac-4c85-ad0a-4a8d8febf8ce None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:58.001715 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ee4c3930-90ac-4c85-ad0a-4a8d8febf8ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:58.076124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 442/872] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:58.088008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e99c-df61-42ee-97ce-ba35dab2e019 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.088296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e99c-df61-42ee-97ce-ba35dab2e019 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.088578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a1e99c-df61-42ee-97ce-ba35dab2e019 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.091542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 431/873] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:58.095802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbc21a-0570-4193-a73d-94d428a8f393 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.096096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbc21a-0570-4193-a73d-94d428a8f393 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.096375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57bbc21a-0570-4193-a73d-94d428a8f393 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.102372 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1e99c-df61-42ee-97ce-ba35dab2e019 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.119191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-57bbc21a-0570-4193-a73d-94d428a8f393 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:58.121240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-57bbc21a-0570-4193-a73d-94d428a8f393 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:58.141731 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.213649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 443/874] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:58.220566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a1e99c-df61-42ee-97ce-ba35dab2e019 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.234403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbc21a-0570-4193-a73d-94d428a8f393 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.264205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ffcdc-4631-427c-aa2e-7276afe74fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.264909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ffcdc-4631-427c-aa2e-7276afe74fd4 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.265195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ffcdc-4631-427c-aa2e-7276afe74fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.265476 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ffcdc-4631-427c-aa2e-7276afe74fd4 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.266602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.268153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ffcdc-4631-427c-aa2e-7276afe74fd4 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.269894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ffcdc-4631-427c-aa2e-7276afe74fd4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.293639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 432/875] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:10:58 2022] POST /identity/v3/services => generated 251 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:10:58.358174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57bbc21a-0570-4193-a73d-94d428a8f393 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.410563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613bbc8-acff-407c-9f0d-d3d6710d2fd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.411405 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613bbc8-acff-407c-9f0d-d3d6710d2fd7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.411695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613bbc8-acff-407c-9f0d-d3d6710d2fd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.411984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6613bbc8-acff-407c-9f0d-d3d6710d2fd7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.426414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6613bbc8-acff-407c-9f0d-d3d6710d2fd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-6613bbc8-acff-407c-9f0d-d3d6710d2fd7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:10:58.435513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:58.436554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:58.437686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 444/876] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:58.445781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ffcdc-4631-427c-aa2e-7276afe74fd4 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.549121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.560950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ffcdc-4631-427c-aa2e-7276afe74fd4 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.597701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611cf8a-a8fc-4d37-8432-d64591ade49b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.599015 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611cf8a-a8fc-4d37-8432-d64591ade49b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.599544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611cf8a-a8fc-4d37-8432-d64591ade49b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.600021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6611cf8a-a8fc-4d37-8432-d64591ade49b None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.600985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611cf8a-a8fc-4d37-8432-d64591ade49b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:58.601498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611cf8a-a8fc-4d37-8432-d64591ade49b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.603093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6611cf8a-a8fc-4d37-8432-d64591ade49b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.617296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 433/877] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:58.618970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613bbc8-acff-407c-9f0d-d3d6710d2fd7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:58.711941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:58.733560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6613bbc8-acff-407c-9f0d-d3d6710d2fd7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:58.770210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb37f6-2481-46f1-bfdb-7d3383aa04e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:58.771007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb37f6-2481-46f1-bfdb-7d3383aa04e9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:58.771294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb37f6-2481-46f1-bfdb-7d3383aa04e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:58.771572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb37f6-2481-46f1-bfdb-7d3383aa04e9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:58.783166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb37f6-2481-46f1-bfdb-7d3383aa04e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:58.784800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb37f6-2481-46f1-bfdb-7d3383aa04e9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-09cb37f6-2481-46f1-bfdb-7d3383aa04e9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:10:58.803713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:58.804636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:58.805786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:10:58.805786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:10:58.805786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:10:58.805786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:58.805786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 445/878] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:58.812894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611cf8a-a8fc-4d37-8432-d64591ade49b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.008313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6611cf8a-a8fc-4d37-8432-d64591ade49b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.025885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695697f-1bab-445a-bf66-bc114a47614f None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.027038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695697f-1bab-445a-bf66-bc114a47614f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.027038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f695697f-1bab-445a-bf66-bc114a47614f None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.033035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 446/879] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:59 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:59.038185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcfda7-7530-4646-8a65-2cf1813b665f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.038470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcfda7-7530-4646-8a65-2cf1813b665f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.038743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0fcfda7-7530-4646-8a65-2cf1813b665f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.071424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1820d82-4bd2-4095-942f-9b1ca165b563 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.072428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1820d82-4bd2-4095-942f-9b1ca165b563 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.073478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1820d82-4bd2-4095-942f-9b1ca165b563 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.074208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1820d82-4bd2-4095-942f-9b1ca165b563 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.075210 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1820d82-4bd2-4095-942f-9b1ca165b563 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:59.076323 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b0fcfda7-7530-4646-8a65-2cf1813b665f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:59.076860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1820d82-4bd2-4095-942f-9b1ca165b563 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.078474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1820d82-4bd2-4095-942f-9b1ca165b563 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.079927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b0fcfda7-7530-4646-8a65-2cf1813b665f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:59.096881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 434/880] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:10:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 284 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:59.103688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1820d82-4bd2-4095-942f-9b1ca165b563 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.128472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.183891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 447/881] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:10:59.206651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcfda7-7530-4646-8a65-2cf1813b665f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.290326 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1820d82-4bd2-4095-942f-9b1ca165b563 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.297262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.351331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2cc2905-4319-4f75-8ee1-8226f1481227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.352263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc2905-4319-4f75-8ee1-8226f1481227 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.352641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc2905-4319-4f75-8ee1-8226f1481227 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.352916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2cc2905-4319-4f75-8ee1-8226f1481227 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.356483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0fcfda7-7530-4646-8a65-2cf1813b665f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.361553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc2905-4319-4f75-8ee1-8226f1481227 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.363040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2cc2905-4319-4f75-8ee1-8226f1481227 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f2cc2905-4319-4f75-8ee1-8226f1481227 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:10:59.378553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:59.379536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:10:59.380490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 435/882] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:10:59.381091 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7d613-e792-45a7-8bbc-ca127d015602 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.381844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7d613-e792-45a7-8bbc-ca127d015602 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.381844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d7d613-e792-45a7-8bbc-ca127d015602 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.383130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 436/883] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:10:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:10:59.385398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7d613-e792-45a7-8bbc-ca127d015602 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.400324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf45b41-33f9-428a-bbbf-45077e68ea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.400987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf45b41-33f9-428a-bbbf-45077e68ea74 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.401307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf45b41-33f9-428a-bbbf-45077e68ea74 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.401612 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf45b41-33f9-428a-bbbf-45077e68ea74 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.402325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf45b41-33f9-428a-bbbf-45077e68ea74 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.403699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baf45b41-33f9-428a-bbbf-45077e68ea74 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.420902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 448/884] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:10:59 2022] POST /identity/v3/services => generated 269 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:10:59.422449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8e794-0f9f-4148-9b2f-7b220f367f9f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.422821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8e794-0f9f-4148-9b2f-7b220f367f9f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.423370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8e794-0f9f-4148-9b2f-7b220f367f9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.467617 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0fb8e794-0f9f-4148-9b2f-7b220f367f9f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:10:59.469780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0fb8e794-0f9f-4148-9b2f-7b220f367f9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:10:59.496150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.547244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d7d613-e792-45a7-8bbc-ca127d015602 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.574387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 449/885] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:10:59 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:10:59.593555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871b9ef-60d1-40db-a28a-e0cdee61f763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.594872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871b9ef-60d1-40db-a28a-e0cdee61f763 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.595484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871b9ef-60d1-40db-a28a-e0cdee61f763 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.596002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8871b9ef-60d1-40db-a28a-e0cdee61f763 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.597694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871b9ef-60d1-40db-a28a-e0cdee61f763 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:59.598033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871b9ef-60d1-40db-a28a-e0cdee61f763 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.599823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8871b9ef-60d1-40db-a28a-e0cdee61f763 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.602402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8e794-0f9f-4148-9b2f-7b220f367f9f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.620757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 437/886] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:10:59.627239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871b9ef-60d1-40db-a28a-e0cdee61f763 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.692419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.748273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8e794-0f9f-4148-9b2f-7b220f367f9f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.761619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8871b9ef-60d1-40db-a28a-e0cdee61f763 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.786119 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388879a-43fb-4411-af3f-57022ddc5719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.786938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388879a-43fb-4411-af3f-57022ddc5719 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.787223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388879a-43fb-4411-af3f-57022ddc5719 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.787499 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1388879a-43fb-4411-af3f-57022ddc5719 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.788160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388879a-43fb-4411-af3f-57022ddc5719 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.789565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1388879a-43fb-4411-af3f-57022ddc5719 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.802774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-1388879a-43fb-4411-af3f-57022ddc5719 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:10:59.805019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:10:59.806081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:10:59.807835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 450/887] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:10:59.813492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388879a-43fb-4411-af3f-57022ddc5719 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:10:59.814645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30945b7d-42da-491a-9366-5061a030283f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.815541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30945b7d-42da-491a-9366-5061a030283f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.816030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30945b7d-42da-491a-9366-5061a030283f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.816531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30945b7d-42da-491a-9366-5061a030283f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.894578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30945b7d-42da-491a-9366-5061a030283f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.896392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30945b7d-42da-491a-9366-5061a030283f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.937761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1388879a-43fb-4411-af3f-57022ddc5719 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:10:59.961449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:10:59.980692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecacda-5f17-401a-9684-7eb683e0f866 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:10:59.981360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecacda-5f17-401a-9684-7eb683e0f866 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:10:59.981700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecacda-5f17-401a-9684-7eb683e0f866 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:10:59.981984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdecacda-5f17-401a-9684-7eb683e0f866 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:10:59.982696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecacda-5f17-401a-9684-7eb683e0f866 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:10:59.982993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecacda-5f17-401a-9684-7eb683e0f866 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:10:59.984470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdecacda-5f17-401a-9684-7eb683e0f866 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:10:59.994721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 451/888] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:10:59.996445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecacda-5f17-401a-9684-7eb683e0f866 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.023989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 438/889] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:00.030069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30945b7d-42da-491a-9366-5061a030283f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.066352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:00.119238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdecacda-5f17-401a-9684-7eb683e0f866 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.153809 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30945b7d-42da-491a-9366-5061a030283f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.158808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9d946-76d9-49f0-9080-91bab930dece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.159477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9d946-76d9-49f0-9080-91bab930dece None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.159753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9d946-76d9-49f0-9080-91bab930dece None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.160201 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff9d946-76d9-49f0-9080-91bab930dece None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.161057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9d946-76d9-49f0-9080-91bab930dece None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:00.161536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9d946-76d9-49f0-9080-91bab930dece None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.163060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff9d946-76d9-49f0-9080-91bab930dece None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:00.173924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 452/890] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Jul 20 19:10:59 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:00.202624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddafd6-3584-4a0b-bd2e-b3807e3049b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.203567 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddafd6-3584-4a0b-bd2e-b3807e3049b9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.203859 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddafd6-3584-4a0b-bd2e-b3807e3049b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.204139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddafd6-3584-4a0b-bd2e-b3807e3049b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.224065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddafd6-3584-4a0b-bd2e-b3807e3049b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.225960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddafd6-3584-4a0b-bd2e-b3807e3049b9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a8ddafd6-3584-4a0b-bd2e-b3807e3049b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:00.247825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:00.248799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:00.249716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:00.249716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:00.249716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:00.249716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:00.252184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 439/891] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:00.258692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9d946-76d9-49f0-9080-91bab930dece None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.320120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:00.382818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff9d946-76d9-49f0-9080-91bab930dece None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.420161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64dfcb-66a4-42ed-8ff8-390be96c229a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.420750 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64dfcb-66a4-42ed-8ff8-390be96c229a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.421036 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64dfcb-66a4-42ed-8ff8-390be96c229a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.421311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af64dfcb-66a4-42ed-8ff8-390be96c229a None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.422069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64dfcb-66a4-42ed-8ff8-390be96c229a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:00.422378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64dfcb-66a4-42ed-8ff8-390be96c229a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.423750 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af64dfcb-66a4-42ed-8ff8-390be96c229a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:00.437494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 453/892] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:00.446589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddafd6-3584-4a0b-bd2e-b3807e3049b9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.533796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:00.559219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddafd6-3584-4a0b-bd2e-b3807e3049b9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.596256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74962955-774b-4e0f-9f33-5c00485dbadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.597064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74962955-774b-4e0f-9f33-5c00485dbadc None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.597350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74962955-774b-4e0f-9f33-5c00485dbadc None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.599281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74962955-774b-4e0f-9f33-5c00485dbadc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.607178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74962955-774b-4e0f-9f33-5c00485dbadc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.608649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74962955-774b-4e0f-9f33-5c00485dbadc None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-74962955-774b-4e0f-9f33-5c00485dbadc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:00.623534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:00.624473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:00.625375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:00.625375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:00.625375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:00.625375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:00.625375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:00.625375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:00.625375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:00.625375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:00.625375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 440/893] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:00.632062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64dfcb-66a4-42ed-8ff8-390be96c229a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.668932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003781a-33bf-478f-b526-d4ecb6d567ef None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.669250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003781a-33bf-478f-b526-d4ecb6d567ef None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.669495 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003781a-33bf-478f-b526-d4ecb6d567ef None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.671261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 441/894] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:00.676387 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b74560-0186-4385-90b1-4d743e1fb1e7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.676676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b74560-0186-4385-90b1-4d743e1fb1e7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.676954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b74560-0186-4385-90b1-4d743e1fb1e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.694903 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-09b74560-0186-4385-90b1-4d743e1fb1e7 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:00.696141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-09b74560-0186-4385-90b1-4d743e1fb1e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:00.744385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af64dfcb-66a4-42ed-8ff8-390be96c229a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.755059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:00.782523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819339b-c94d-43df-ad9a-d50aed6cc9cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.783201 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819339b-c94d-43df-ad9a-d50aed6cc9cc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.783487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819339b-c94d-43df-ad9a-d50aed6cc9cc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.783767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8819339b-c94d-43df-ad9a-d50aed6cc9cc None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.784561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819339b-c94d-43df-ad9a-d50aed6cc9cc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:00.784874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819339b-c94d-43df-ad9a-d50aed6cc9cc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.786243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 442/895] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:00.789978 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8819339b-c94d-43df-ad9a-d50aed6cc9cc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:00.803305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 454/896] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:00.810202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b74560-0186-4385-90b1-4d743e1fb1e7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.810744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819339b-c94d-43df-ad9a-d50aed6cc9cc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:00.893816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:00.928991 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b74560-0186-4385-90b1-4d743e1fb1e7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.939226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8819339b-c94d-43df-ad9a-d50aed6cc9cc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:00.968656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a342ac-4e0d-4e00-8ffa-2ab9cccbf206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.969800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a342ac-4e0d-4e00-8ffa-2ab9cccbf206 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.970293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a342ac-4e0d-4e00-8ffa-2ab9cccbf206 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.970878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a342ac-4e0d-4e00-8ffa-2ab9cccbf206 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.972005 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a342ac-4e0d-4e00-8ffa-2ab9cccbf206 None admin] RBAC: Adding query filter params (role.id=3e0a48dd676e4ee0b8f3cad1dbe68a6b, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=cbd4016436b54a0fb8c74c35273957a0) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:00.972620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a342ac-4e0d-4e00-8ffa-2ab9cccbf206 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.974557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a342ac-4e0d-4e00-8ffa-2ab9cccbf206 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:00.981252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f7870-c5d2-4946-90a6-f0769d65d1ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:00.981821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 443/897] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/role_assignments?user.id=cbd4016436b54a0fb8c74c35273957a0&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=3e0a48dd676e4ee0b8f3cad1dbe68a6b => generated 259 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:00.982085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f7870-c5d2-4946-90a6-f0769d65d1ba None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:00.982364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f7870-c5d2-4946-90a6-f0769d65d1ba None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:00.982637 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083f7870-c5d2-4946-90a6-f0769d65d1ba None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:00.983289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f7870-c5d2-4946-90a6-f0769d65d1ba None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:00.984633 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083f7870-c5d2-4946-90a6-f0769d65d1ba None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-083f7870-c5d2-4946-90a6-f0769d65d1ba None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:01.005544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:01.006622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:01.007476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:01.016067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 455/898] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:01.017946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a342ac-4e0d-4e00-8ffa-2ab9cccbf206 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.135037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a342ac-4e0d-4e00-8ffa-2ab9cccbf206 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.174120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1dde8c-f7e3-413c-b682-15e867b9403a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.174796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dde8c-f7e3-413c-b682-15e867b9403a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.175078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dde8c-f7e3-413c-b682-15e867b9403a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.175354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1dde8c-f7e3-413c-b682-15e867b9403a None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.176063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1dde8c-f7e3-413c-b682-15e867b9403a None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:01.176373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1dde8c-f7e3-413c-b682-15e867b9403a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.177768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1dde8c-f7e3-413c-b682-15e867b9403a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.180130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.184508 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 444/899] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:01.190348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f7870-c5d2-4946-90a6-f0769d65d1ba None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.229696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.308620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083f7870-c5d2-4946-90a6-f0769d65d1ba None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.351875 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199c59a-a20f-474b-a7a9-64375edf04c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.352909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199c59a-a20f-474b-a7a9-64375edf04c2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.353421 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199c59a-a20f-474b-a7a9-64375edf04c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.354605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7199c59a-a20f-474b-a7a9-64375edf04c2 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.355728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199c59a-a20f-474b-a7a9-64375edf04c2 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:01.356326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199c59a-a20f-474b-a7a9-64375edf04c2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.358037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7199c59a-a20f-474b-a7a9-64375edf04c2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.369033 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 456/900] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:01.374658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1dde8c-f7e3-413c-b682-15e867b9403a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.486983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1dde8c-f7e3-413c-b682-15e867b9403a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.523435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8b970e-77e6-4584-bb00-4dc8454d18f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.524411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b970e-77e6-4584-bb00-4dc8454d18f1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.524880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b970e-77e6-4584-bb00-4dc8454d18f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.525340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8b970e-77e6-4584-bb00-4dc8454d18f1 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.527648 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b970e-77e6-4584-bb00-4dc8454d18f1 None admin] RBAC: Adding query filter params (interface=public, service_id=432440190a1d4613b9e8de425f523628, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:01.528184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b970e-77e6-4584-bb00-4dc8454d18f1 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.529781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8b970e-77e6-4584-bb00-4dc8454d18f1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.540238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 445/901] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/endpoints?service_id=432440190a1d4613b9e8de425f523628&interface=public®ion_id=RegionOne => generated 194 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:01.541893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199c59a-a20f-474b-a7a9-64375edf04c2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.646540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.653848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7199c59a-a20f-474b-a7a9-64375edf04c2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.684594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660136-9f10-43a7-9cec-566a9cd901fd None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.684898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660136-9f10-43a7-9cec-566a9cd901fd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.685165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be660136-9f10-43a7-9cec-566a9cd901fd None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.687037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 446/902] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:01.689457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdeab18-8eef-426b-97b3-6a26c649cf22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.690159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeab18-8eef-426b-97b3-6a26c649cf22 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.690445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeab18-8eef-426b-97b3-6a26c649cf22 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.690719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdeab18-8eef-426b-97b3-6a26c649cf22 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.691402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdeab18-8eef-426b-97b3-6a26c649cf22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.692023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe287805-d6ff-49a8-a59d-bfedaabed4da None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.692309 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe287805-d6ff-49a8-a59d-bfedaabed4da None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.692587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe287805-d6ff-49a8-a59d-bfedaabed4da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.692758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdeab18-8eef-426b-97b3-6a26c649cf22 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.699909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 457/903] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/services => generated 1268 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:11:01.710666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fe287805-d6ff-49a8-a59d-bfedaabed4da None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:01.711880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fe287805-d6ff-49a8-a59d-bfedaabed4da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:01.780534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:01.790023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 447/904] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:01 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:01.813161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdeab18-8eef-426b-97b3-6a26c649cf22 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:01.926200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdeab18-8eef-426b-97b3-6a26c649cf22 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:01.962752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b888d8-756f-4c0a-8788-54fa5b3d9e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:01.964678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b888d8-756f-4c0a-8788-54fa5b3d9e73 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:01.965278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b888d8-756f-4c0a-8788-54fa5b3d9e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:01.965794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b888d8-756f-4c0a-8788-54fa5b3d9e73 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:01.966640 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b888d8-756f-4c0a-8788-54fa5b3d9e73 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:01.968282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b888d8-756f-4c0a-8788-54fa5b3d9e73 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-17b888d8-756f-4c0a-8788-54fa5b3d9e73 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:01.982814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:01.983799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:01.984686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:01.987265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 458/905] 10.222.0.69 () {56 vars in 1132 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:01.993341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe287805-d6ff-49a8-a59d-bfedaabed4da None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.100837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe287805-d6ff-49a8-a59d-bfedaabed4da None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.136450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec3390-94f6-4a0a-bf36-34dca3351183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.137270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec3390-94f6-4a0a-bf36-34dca3351183 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.137582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec3390-94f6-4a0a-bf36-34dca3351183 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.137866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eec3390-94f6-4a0a-bf36-34dca3351183 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.138573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec3390-94f6-4a0a-bf36-34dca3351183 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:02.138874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec3390-94f6-4a0a-bf36-34dca3351183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.140251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eec3390-94f6-4a0a-bf36-34dca3351183 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.145551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 448/906] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:01 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:02.151669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b888d8-756f-4c0a-8788-54fa5b3d9e73 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.179663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.267562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b888d8-756f-4c0a-8788-54fa5b3d9e73 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.303569 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0877f0-b230-48c5-a8c2-dfffeece5432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.304246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0877f0-b230-48c5-a8c2-dfffeece5432 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.304530 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0877f0-b230-48c5-a8c2-dfffeece5432 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.304810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0877f0-b230-48c5-a8c2-dfffeece5432 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.305542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0877f0-b230-48c5-a8c2-dfffeece5432 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:02.305861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0877f0-b230-48c5-a8c2-dfffeece5432 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.307962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0877f0-b230-48c5-a8c2-dfffeece5432 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.319078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 459/907] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:02.326254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec3390-94f6-4a0a-bf36-34dca3351183 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.381151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325ae30-219f-4c05-bd8e-caf428fd99d6 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.381338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325ae30-219f-4c05-bd8e-caf428fd99d6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.381669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8325ae30-219f-4c05-bd8e-caf428fd99d6 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.383295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 460/908] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:02.389250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1728a89-f5e8-49b9-9b2e-722137311260 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.389568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1728a89-f5e8-49b9-9b2e-722137311260 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.389689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.389865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1728a89-f5e8-49b9-9b2e-722137311260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.410185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b1728a89-f5e8-49b9-9b2e-722137311260 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:02.411381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b1728a89-f5e8-49b9-9b2e-722137311260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:02.440542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eec3390-94f6-4a0a-bf36-34dca3351183 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.490660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ddddf-5eb7-41c9-b78a-fc79555cf135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.491330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ddddf-5eb7-41c9-b78a-fc79555cf135 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.491615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ddddf-5eb7-41c9-b78a-fc79555cf135 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.491888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d39ddddf-5eb7-41c9-b78a-fc79555cf135 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.492778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ddddf-5eb7-41c9-b78a-fc79555cf135 None admin] RBAC: Adding query filter params (interface=public, service_id=e5774141a451493bb458c52931172eaf, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:02.493089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ddddf-5eb7-41c9-b78a-fc79555cf135 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.494493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d39ddddf-5eb7-41c9-b78a-fc79555cf135 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.500842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 449/909] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/endpoints?service_id=e5774141a451493bb458c52931172eaf&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:02.501718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 461/910] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:02.506205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ddddf-5eb7-41c9-b78a-fc79555cf135 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.518113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1728a89-f5e8-49b9-9b2e-722137311260 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:02.540221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.625133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d39ddddf-5eb7-41c9-b78a-fc79555cf135 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.641940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1728a89-f5e8-49b9-9b2e-722137311260 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:02.654057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:02.668523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb93ad9-32af-4757-b941-b2665f6af6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.669690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93ad9-32af-4757-b941-b2665f6af6ed None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.670293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93ad9-32af-4757-b941-b2665f6af6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.670789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb93ad9-32af-4757-b941-b2665f6af6ed None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.671676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb93ad9-32af-4757-b941-b2665f6af6ed None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.673459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb93ad9-32af-4757-b941-b2665f6af6ed None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.680218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 450/911] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:02 2022] GET /identity/v3/services => generated 1268 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:11:02.681761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822744ed-227d-417c-872b-191cbbe91a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:02.683017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822744ed-227d-417c-872b-191cbbe91a09 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:02.683243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822744ed-227d-417c-872b-191cbbe91a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:02.683513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822744ed-227d-417c-872b-191cbbe91a09 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:02.684301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822744ed-227d-417c-872b-191cbbe91a09 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:02.685690 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822744ed-227d-417c-872b-191cbbe91a09 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:02.702462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 462/912] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:02 2022] POST /identity/v3/services => generated 241 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:11:03.215962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623bfcf-bedb-4714-aa0b-b54971e05603 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.216851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623bfcf-bedb-4714-aa0b-b54971e05603 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.217290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b623bfcf-bedb-4714-aa0b-b54971e05603 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.219067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 451/913] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:03.224878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449783c-1d2f-4c22-89b5-d14eb31b56e2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.225350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449783c-1d2f-4c22-89b5-d14eb31b56e2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.225816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8449783c-1d2f-4c22-89b5-d14eb31b56e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.243935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8449783c-1d2f-4c22-89b5-d14eb31b56e2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:03.245903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8449783c-1d2f-4c22-89b5-d14eb31b56e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:03.331438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 463/914] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:03.357051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623bfcf-bedb-4714-aa0b-b54971e05603 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.462848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b623bfcf-bedb-4714-aa0b-b54971e05603 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.507321 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4af1-9594-4133-9fcc-e6c1fb90f6ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.508048 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4af1-9594-4133-9fcc-e6c1fb90f6ed None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.508322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4af1-9594-4133-9fcc-e6c1fb90f6ed None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.508590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2a4af1-9594-4133-9fcc-e6c1fb90f6ed None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.519070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2a4af1-9594-4133-9fcc-e6c1fb90f6ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-9c2a4af1-9594-4133-9fcc-e6c1fb90f6ed None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:03.527541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:03.528572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:03.529486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 452/915] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:03.536965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449783c-1d2f-4c22-89b5-d14eb31b56e2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.540470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.654596 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8449783c-1d2f-4c22-89b5-d14eb31b56e2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.689287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf8c954-7edd-4d33-adfa-7f70bd6f264c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.689984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8c954-7edd-4d33-adfa-7f70bd6f264c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.690265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8c954-7edd-4d33-adfa-7f70bd6f264c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.690539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf8c954-7edd-4d33-adfa-7f70bd6f264c None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.691276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8c954-7edd-4d33-adfa-7f70bd6f264c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:03.691578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8c954-7edd-4d33-adfa-7f70bd6f264c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.692939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf8c954-7edd-4d33-adfa-7f70bd6f264c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.702199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 464/916] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:03.710596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4af1-9594-4133-9fcc-e6c1fb90f6ed None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.761107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.824448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2a4af1-9594-4133-9fcc-e6c1fb90f6ed None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:03.859853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcb0da-20ba-4328-bea4-003e42273393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:03.860650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcb0da-20ba-4328-bea4-003e42273393 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.860930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcb0da-20ba-4328-bea4-003e42273393 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.861204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46dcb0da-20ba-4328-bea4-003e42273393 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.864702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2be1a-719b-4aac-a3eb-a09e89c35fe5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.864990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2be1a-719b-4aac-a3eb-a09e89c35fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.865247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed2be1a-719b-4aac-a3eb-a09e89c35fe5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.867106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 465/917] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:03.875031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcb0da-20ba-4328-bea4-003e42273393 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:03.875525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8528ed-ec8f-4112-8046-796446575d42 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:03.876027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8528ed-ec8f-4112-8046-796446575d42 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:03.876500 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8528ed-ec8f-4112-8046-796446575d42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:03.876618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46dcb0da-20ba-4328-bea4-003e42273393 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:03.894516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2a8528ed-ec8f-4112-8046-796446575d42 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-46dcb0da-20ba-4328-bea4-003e42273393 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:03.894948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:03.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:03.896844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:03.896844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:03.896844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:03.896844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:03.896844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 453/918] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:03.898840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2a8528ed-ec8f-4112-8046-796446575d42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:03.909245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcb0da-20ba-4328-bea4-003e42273393 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:03.966927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:03.985433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 466/919] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:04.001614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8528ed-ec8f-4112-8046-796446575d42 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.020414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.024499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46dcb0da-20ba-4328-bea4-003e42273393 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.064257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2433cf-4871-4554-94e4-95ffd9cf13e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.064928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2433cf-4871-4554-94e4-95ffd9cf13e9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.065212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2433cf-4871-4554-94e4-95ffd9cf13e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.065493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2433cf-4871-4554-94e4-95ffd9cf13e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.066236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2433cf-4871-4554-94e4-95ffd9cf13e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:04.066573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2433cf-4871-4554-94e4-95ffd9cf13e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.068127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2433cf-4871-4554-94e4-95ffd9cf13e9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.081546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 454/920] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:04.087257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2433cf-4871-4554-94e4-95ffd9cf13e9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.115326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8528ed-ec8f-4112-8046-796446575d42 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.153257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a215f1-95f7-425a-827a-588f43fde101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.161724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a215f1-95f7-425a-827a-588f43fde101 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.161724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a215f1-95f7-425a-827a-588f43fde101 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.161724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a215f1-95f7-425a-827a-588f43fde101 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.161724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a215f1-95f7-425a-827a-588f43fde101 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.161724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a215f1-95f7-425a-827a-588f43fde101 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b5a215f1-95f7-425a-827a-588f43fde101 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:04.170688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:04.171741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 20 19:11:04.172641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:04.180322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 467/921] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:04.182319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a215f1-95f7-425a-827a-588f43fde101 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.198642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2433cf-4871-4554-94e4-95ffd9cf13e9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.236858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965fb3-00e8-4f86-83b4-f2b0bc1b8880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.238480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965fb3-00e8-4f86-83b4-f2b0bc1b8880 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.238966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965fb3-00e8-4f86-83b4-f2b0bc1b8880 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.239428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22965fb3-00e8-4f86-83b4-f2b0bc1b8880 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.245166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.250246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22965fb3-00e8-4f86-83b4-f2b0bc1b8880 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.252450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22965fb3-00e8-4f86-83b4-f2b0bc1b8880 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-22965fb3-00e8-4f86-83b4-f2b0bc1b8880 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:04.269413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:04.270596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:04.271592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:04.271592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:04.271592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:04.271592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:04.271592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:04.271592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:04.271592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:04.271592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:04.271592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:04.271592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:04.279995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 455/922] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:04.281128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965fb3-00e8-4f86-83b4-f2b0bc1b8880 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.288731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a215f1-95f7-425a-827a-588f43fde101 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.309467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.331252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04250f-7e12-4849-8f29-628a3cd57ec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.332255 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04250f-7e12-4849-8f29-628a3cd57ec6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.333276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04250f-7e12-4849-8f29-628a3cd57ec6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.333761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa04250f-7e12-4849-8f29-628a3cd57ec6 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.334641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04250f-7e12-4849-8f29-628a3cd57ec6 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:04.335145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04250f-7e12-4849-8f29-628a3cd57ec6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.336795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa04250f-7e12-4849-8f29-628a3cd57ec6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.347234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 468/923] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:04.349141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04250f-7e12-4849-8f29-628a3cd57ec6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.420666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22965fb3-00e8-4f86-83b4-f2b0bc1b8880 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.460377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa04250f-7e12-4849-8f29-628a3cd57ec6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.463471 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26b240-f112-46ac-95a1-70b03ae9097f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.464454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26b240-f112-46ac-95a1-70b03ae9097f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.465139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26b240-f112-46ac-95a1-70b03ae9097f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.465698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d26b240-f112-46ac-95a1-70b03ae9097f None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.467481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26b240-f112-46ac-95a1-70b03ae9097f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:04.468324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26b240-f112-46ac-95a1-70b03ae9097f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.470644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d26b240-f112-46ac-95a1-70b03ae9097f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.486006 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.501265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf6e08-d825-46ec-8520-e015394cfe9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.501265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf6e08-d825-46ec-8520-e015394cfe9e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.501265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf6e08-d825-46ec-8520-e015394cfe9e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.501265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbf6e08-d825-46ec-8520-e015394cfe9e None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.501749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf6e08-d825-46ec-8520-e015394cfe9e None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:04.501749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf6e08-d825-46ec-8520-e015394cfe9e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.501749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbf6e08-d825-46ec-8520-e015394cfe9e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.502418 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 456/924] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:04.503682 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26b240-f112-46ac-95a1-70b03ae9097f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.511462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 469/925] 10.222.0.69 () {56 vars in 1114 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:04.512688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf6e08-d825-46ec-8520-e015394cfe9e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.594728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.615408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbf6e08-d825-46ec-8520-e015394cfe9e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.624561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d26b240-f112-46ac-95a1-70b03ae9097f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.651415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb129f5f-0bcc-4c4e-9cad-ece6392ee4cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.652169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb129f5f-0bcc-4c4e-9cad-ece6392ee4cc None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.652513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb129f5f-0bcc-4c4e-9cad-ece6392ee4cc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.652871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb129f5f-0bcc-4c4e-9cad-ece6392ee4cc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.653859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb129f5f-0bcc-4c4e-9cad-ece6392ee4cc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.655296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb129f5f-0bcc-4c4e-9cad-ece6392ee4cc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.665240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a65ae2-854f-4f6e-aaca-66ad8da3f64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.666185 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a65ae2-854f-4f6e-aaca-66ad8da3f64b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.667045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a65ae2-854f-4f6e-aaca-66ad8da3f64b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.667564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a65ae2-854f-4f6e-aaca-66ad8da3f64b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.684354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a65ae2-854f-4f6e-aaca-66ad8da3f64b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.684354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a65ae2-854f-4f6e-aaca-66ad8da3f64b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.684900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 470/926] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-22a65ae2-854f-4f6e-aaca-66ad8da3f64b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:04.702331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:04.703352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:04.704735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:04.704735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:04.704735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:04.704735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:04.704735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 457/927] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:04.709959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb129f5f-0bcc-4c4e-9cad-ece6392ee4cc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.769482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:04.771559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51527176-b7bd-4156-b5d9-d7f8216e3ae5 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.771852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51527176-b7bd-4156-b5d9-d7f8216e3ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.772130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51527176-b7bd-4156-b5d9-d7f8216e3ae5 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.774039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 458/928] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:04.780178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13503eb-0499-4a87-8809-e02387fe2d2f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.780465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13503eb-0499-4a87-8809-e02387fe2d2f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.780743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13503eb-0499-4a87-8809-e02387fe2d2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.799067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c13503eb-0499-4a87-8809-e02387fe2d2f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:04.800296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c13503eb-0499-4a87-8809-e02387fe2d2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:04.860313 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb129f5f-0bcc-4c4e-9cad-ece6392ee4cc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:04.889611 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 459/929] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:04 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:04.911062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed7f28-cfff-4f92-9042-f2a123ba0c1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:04.911740 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed7f28-cfff-4f92-9042-f2a123ba0c1d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:04.912296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed7f28-cfff-4f92-9042-f2a123ba0c1d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:04.912296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ed7f28-cfff-4f92-9042-f2a123ba0c1d None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:04.913029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed7f28-cfff-4f92-9042-f2a123ba0c1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:04.913338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed7f28-cfff-4f92-9042-f2a123ba0c1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:04.914065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13503eb-0499-4a87-8809-e02387fe2d2f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.914743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ed7f28-cfff-4f92-9042-f2a123ba0c1d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:04.929281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 471/930] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:04.935266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed7f28-cfff-4f92-9042-f2a123ba0c1d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:04.987007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.032983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13503eb-0499-4a87-8809-e02387fe2d2f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.049579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.052817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ed7f28-cfff-4f92-9042-f2a123ba0c1d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.072430 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d68fc1b-fa35-4201-83b5-d986d0496c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.073819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68fc1b-fa35-4201-83b5-d986d0496c05 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.074140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68fc1b-fa35-4201-83b5-d986d0496c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.074414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d68fc1b-fa35-4201-83b5-d986d0496c05 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.075129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68fc1b-fa35-4201-83b5-d986d0496c05 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.076499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d68fc1b-fa35-4201-83b5-d986d0496c05 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.089749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b115c9-fecb-4c80-9bc0-77acf93c1145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6d68fc1b-fa35-4201-83b5-d986d0496c05 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:05.089959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:05.090866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 20 19:11:05.092318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:05.093154 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b115c9-fecb-4c80-9bc0-77acf93c1145 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.093154 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b115c9-fecb-4c80-9bc0-77acf93c1145 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.093154 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b115c9-fecb-4c80-9bc0-77acf93c1145 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.093350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 460/931] 10.222.0.69 () {56 vars in 1132 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:05.093350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a9398-80a4-406f-966e-f50019987081 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.093350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a9398-80a4-406f-966e-f50019987081 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.093350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb6a9398-80a4-406f-966e-f50019987081 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.094713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 461/932] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:05.099049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a9398-80a4-406f-966e-f50019987081 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.104056 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b115c9-fecb-4c80-9bc0-77acf93c1145 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.105658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b115c9-fecb-4c80-9bc0-77acf93c1145 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f4b115c9-fecb-4c80-9bc0-77acf93c1145 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:05.120526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:05.121449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 472/933] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5ce45-8047-4950-a415-25559b1347cf None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.127939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5ce45-8047-4950-a415-25559b1347cf None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.128573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb5ce45-8047-4950-a415-25559b1347cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.141657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bcb5ce45-8047-4950-a415-25559b1347cf None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:05.142851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bcb5ce45-8047-4950-a415-25559b1347cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:05.221535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 473/934] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:05.221535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5ce45-8047-4950-a415-25559b1347cf None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.224026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb6a9398-80a4-406f-966e-f50019987081 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.250422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.264176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58edbc-a3d9-46ff-9c31-d376b4fff0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.264852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58edbc-a3d9-46ff-9c31-d376b4fff0ee None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.265140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58edbc-a3d9-46ff-9c31-d376b4fff0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.265419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58edbc-a3d9-46ff-9c31-d376b4fff0ee None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.266164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58edbc-a3d9-46ff-9c31-d376b4fff0ee None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:05.266476 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58edbc-a3d9-46ff-9c31-d376b4fff0ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.267940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58edbc-a3d9-46ff-9c31-d376b4fff0ee None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.273294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 462/935] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:05.274438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58edbc-a3d9-46ff-9c31-d376b4fff0ee None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.335125 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.348095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb5ce45-8047-4950-a415-25559b1347cf None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.383006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58edbc-a3d9-46ff-9c31-d376b4fff0ee None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.386986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02430c-cc87-4cd5-a219-1553d34c36aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.387804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02430c-cc87-4cd5-a219-1553d34c36aa None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.388085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02430c-cc87-4cd5-a219-1553d34c36aa None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.388356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb02430c-cc87-4cd5-a219-1553d34c36aa None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.389141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02430c-cc87-4cd5-a219-1553d34c36aa None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:05.389450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02430c-cc87-4cd5-a219-1553d34c36aa None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.391034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb02430c-cc87-4cd5-a219-1553d34c36aa None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.404188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 474/936] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:05.409587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02430c-cc87-4cd5-a219-1553d34c36aa None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.420225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186138b-6076-4625-bfe1-90e2dc46bc68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.421035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186138b-6076-4625-bfe1-90e2dc46bc68 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.421323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186138b-6076-4625-bfe1-90e2dc46bc68 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.421626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d186138b-6076-4625-bfe1-90e2dc46bc68 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.422284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186138b-6076-4625-bfe1-90e2dc46bc68 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.423765 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d186138b-6076-4625-bfe1-90e2dc46bc68 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-d186138b-6076-4625-bfe1-90e2dc46bc68 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:05.439657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:05.440734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:05.442288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 463/937] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:05.443135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186138b-6076-4625-bfe1-90e2dc46bc68 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.495283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.527985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb02430c-cc87-4cd5-a219-1553d34c36aa None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.548278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d186138b-6076-4625-bfe1-90e2dc46bc68 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.563545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3b521-ced7-4f3b-bdce-426fa2a41506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.564209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3b521-ced7-4f3b-bdce-426fa2a41506 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.564492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3b521-ced7-4f3b-bdce-426fa2a41506 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.564759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e3b521-ced7-4f3b-bdce-426fa2a41506 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.565456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3b521-ced7-4f3b-bdce-426fa2a41506 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:05.565782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3b521-ced7-4f3b-bdce-426fa2a41506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.567131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e3b521-ced7-4f3b-bdce-426fa2a41506 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.578618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 475/938] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:05.578618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3b521-ced7-4f3b-bdce-426fa2a41506 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.588690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c9012-4dcc-4dc0-85d8-7351957d3054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.589532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c9012-4dcc-4dc0-85d8-7351957d3054 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.589808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c9012-4dcc-4dc0-85d8-7351957d3054 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.590085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-982c9012-4dcc-4dc0-85d8-7351957d3054 None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/cbd4016436b54a0fb8c74c35273957a0/roles/3e0a48dd676e4ee0b8f3cad1dbe68a6b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.593637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c9012-4dcc-4dc0-85d8-7351957d3054 None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=cbd4016436b54a0fb8c74c35273957a0, role_id=3e0a48dd676e4ee0b8f3cad1dbe68a6b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.595090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-982c9012-4dcc-4dc0-85d8-7351957d3054 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.596084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.610652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 464/939] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:05 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/cbd4016436b54a0fb8c74c35273957a0/roles/3e0a48dd676e4ee0b8f3cad1dbe68a6b => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:11:05.611528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c9012-4dcc-4dc0-85d8-7351957d3054 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.719359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e3b521-ced7-4f3b-bdce-426fa2a41506 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.746457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-982c9012-4dcc-4dc0-85d8-7351957d3054 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.759091 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.759426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c71c9e-86fb-427e-8d74-0c6f405900fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.761299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71c9e-86fb-427e-8d74-0c6f405900fa None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.761681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71c9e-86fb-427e-8d74-0c6f405900fa None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.762188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5c71c9e-86fb-427e-8d74-0c6f405900fa None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.762931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c71c9e-86fb-427e-8d74-0c6f405900fa None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:05.763240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c71c9e-86fb-427e-8d74-0c6f405900fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.764866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5c71c9e-86fb-427e-8d74-0c6f405900fa None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.771758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 476/940] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:05.779779 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c71c9e-86fb-427e-8d74-0c6f405900fa None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:05.791266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35b424-ed46-4f3f-8830-a83ec5f4b399 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.793093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35b424-ed46-4f3f-8830-a83ec5f4b399 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.793530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35b424-ed46-4f3f-8830-a83ec5f4b399 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.793968 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d35b424-ed46-4f3f-8830-a83ec5f4b399 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.794803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35b424-ed46-4f3f-8830-a83ec5f4b399 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.796266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d35b424-ed46-4f3f-8830-a83ec5f4b399 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.823370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 465/941] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:05 2022] POST /identity/v3/endpoints => generated 347 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:05.881482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:05.907614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5c71c9e-86fb-427e-8d74-0c6f405900fa None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:05.955199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32740b-d854-4aa4-9625-01cee33d934a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:05.955881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32740b-d854-4aa4-9625-01cee33d934a None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:05.956162 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32740b-d854-4aa4-9625-01cee33d934a None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:05.956434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c32740b-d854-4aa4-9625-01cee33d934a None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:05.957141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32740b-d854-4aa4-9625-01cee33d934a None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:05.957443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32740b-d854-4aa4-9625-01cee33d934a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:05.958847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c32740b-d854-4aa4-9625-01cee33d934a None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:05.969993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 477/942] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:05.971924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35b424-ed46-4f3f-8830-a83ec5f4b399 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.078538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d35b424-ed46-4f3f-8830-a83ec5f4b399 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.114361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20793102-fd74-4016-99e5-4bdfd13a4200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.115026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793102-fd74-4016-99e5-4bdfd13a4200 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.115313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793102-fd74-4016-99e5-4bdfd13a4200 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.115590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20793102-fd74-4016-99e5-4bdfd13a4200 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.116377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20793102-fd74-4016-99e5-4bdfd13a4200 None admin] RBAC: Adding query filter params (interface=public, service_id=c1d906f25dc0453d89b7b975a79137c9, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:06.116688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20793102-fd74-4016-99e5-4bdfd13a4200 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.121701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20793102-fd74-4016-99e5-4bdfd13a4200 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.126166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 466/943] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:05 2022] GET /identity/v3/endpoints?service_id=c1d906f25dc0453d89b7b975a79137c9&interface=public®ion_id=RegionOne => generated 194 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:06.133550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32740b-d854-4aa4-9625-01cee33d934a None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:06.236767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c32740b-d854-4aa4-9625-01cee33d934a None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:06.272389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d5f7d-8272-4bb5-af8c-3207e63980db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:06.273052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d5f7d-8272-4bb5-af8c-3207e63980db None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.273333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d5f7d-8272-4bb5-af8c-3207e63980db None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.273841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d5f7d-8272-4bb5-af8c-3207e63980db None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.274322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d5f7d-8272-4bb5-af8c-3207e63980db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:06.275701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d5f7d-8272-4bb5-af8c-3207e63980db None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:06.289678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 478/944] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:06 2022] GET /identity/v3/services => generated 1497 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:11:06.905436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd882b-5891-42e4-bbfe-ca17e77ecd46 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.906020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd882b-5891-42e4-bbfe-ca17e77ecd46 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.906020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd882b-5891-42e4-bbfe-ca17e77ecd46 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.907823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 467/945] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:06.913304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7404f7b-5953-40b2-9e45-a4e0c48fc1c1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:06.913817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7404f7b-5953-40b2-9e45-a4e0c48fc1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:06.913940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7404f7b-5953-40b2-9e45-a4e0c48fc1c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:06.919512 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:06.932984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c7404f7b-5953-40b2-9e45-a4e0c48fc1c1 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:06.934201 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c7404f7b-5953-40b2-9e45-a4e0c48fc1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:07.007267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 479/946] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:07.031355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd882b-5891-42e4-bbfe-ca17e77ecd46 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.063560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:07.140471 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66cd882b-5891-42e4-bbfe-ca17e77ecd46 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.174812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.175401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.175685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.175959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.176660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:07.178120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:07.179471 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: WARNING keystone.server.flask.common [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 19:11:07.197468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.sql.core [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 19:11:07.197468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 19:11:07.197468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [parameters: {'id': '0fa1d5619d484905b70cc65f37a715ad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 19:11:07.197468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80211) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0fa1d5619d484905b70cc65f37a715ad.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0fa1d5619d484905b70cc65f37a715ad. Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:11:07.206339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result.read() Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 20 19:11:07.207177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application yield resource Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise self.value Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application session.commit() Jul 20 19:11:07.208143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.session.flush() Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application compat.raise_( Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 19:11:07.209050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 19:11:07.211262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application result.read() Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:11:07.213176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [parameters: {'id': '0fa1d5619d484905b70cc65f37a715ad', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:07.215365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:07.216988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:07.226444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:07.226444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 20 19:11:07.226444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 20 19:11:07.226444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:07.226444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:07.226444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 20 19:11:07.226444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 20 19:11:07.226444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0fa1d5619d484905b70cc65f37a715ad. Jul 20 19:11:07.226444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:07.226444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 468/947] 10.222.0.69 () {60 vars in 1140 bytes} [Wed Jul 20 19:11:07 2022] POST /identity/v3/projects => generated 262 bytes in 177 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:07.227164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7404f7b-5953-40b2-9e45-a4e0c48fc1c1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.331878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7404f7b-5953-40b2-9e45-a4e0c48fc1c1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.364342 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:07.371380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b511db-2b9a-443c-8cbf-b8c0a02184df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.372419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b511db-2b9a-443c-8cbf-b8c0a02184df None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.372904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b511db-2b9a-443c-8cbf-b8c0a02184df None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.373375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b511db-2b9a-443c-8cbf-b8c0a02184df None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.380988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b511db-2b9a-443c-8cbf-b8c0a02184df None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:07.382739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57b511db-2b9a-443c-8cbf-b8c0a02184df None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-57b511db-2b9a-443c-8cbf-b8c0a02184df None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:07.397254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:07.398226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:07.399108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:07.399108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:07.399108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:07.399108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:07.399108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:07.399108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:07.399108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:07.399108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:07.401524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 480/948] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:07.413572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:07.504438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:07.524481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec7dc5-e537-4f02-a1ff-8b74c4d93ef3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:07.560850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f3fd91-9b98-4599-b294-7909b6fb942d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:07.561535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3fd91-9b98-4599-b294-7909b6fb942d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.561828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3fd91-9b98-4599-b294-7909b6fb942d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.562103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f3fd91-9b98-4599-b294-7909b6fb942d None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.562855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3fd91-9b98-4599-b294-7909b6fb942d None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:07.563165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3fd91-9b98-4599-b294-7909b6fb942d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:07.564783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f3fd91-9b98-4599-b294-7909b6fb942d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:07.587856 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 469/949] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:07.855419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af212a-6ce3-4e2d-8618-0b6ae4ff9260 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.856191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af212a-6ce3-4e2d-8618-0b6ae4ff9260 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.856674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36af212a-6ce3-4e2d-8618-0b6ae4ff9260 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.858618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 481/950] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:07.864217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33def065-ef6a-4103-81f8-34f6ef9856af None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.864784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33def065-ef6a-4103-81f8-34f6ef9856af None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.865291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33def065-ef6a-4103-81f8-34f6ef9856af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.884847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-33def065-ef6a-4103-81f8-34f6ef9856af None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:07.887478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-33def065-ef6a-4103-81f8-34f6ef9856af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:07.972135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d344d-38d3-479a-95e9-942fc194e508 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.972645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 470/951] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:07.973379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d344d-38d3-479a-95e9-942fc194e508 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.973379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233d344d-38d3-479a-95e9-942fc194e508 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.974533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 482/952] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:07.980715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9534f8-e91d-4314-9f6f-94977cc397de None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:07.981009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9534f8-e91d-4314-9f6f-94977cc397de None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:07.981292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b9534f8-e91d-4314-9f6f-94977cc397de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:07.999148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2b9534f8-e91d-4314-9f6f-94977cc397de None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:07.999751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d344d-38d3-479a-95e9-942fc194e508 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.000353 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2b9534f8-e91d-4314-9f6f-94977cc397de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:08.093587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 471/953] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:08.116488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9534f8-e91d-4314-9f6f-94977cc397de None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.120046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233d344d-38d3-479a-95e9-942fc194e508 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.126078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.159699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d321af-30b3-4c8e-9d42-559064991a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.160910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d321af-30b3-4c8e-9d42-559064991a99 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.161437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d321af-30b3-4c8e-9d42-559064991a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.161981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d321af-30b3-4c8e-9d42-559064991a99 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.174157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d321af-30b3-4c8e-9d42-559064991a99 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-44d321af-30b3-4c8e-9d42-559064991a99 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:08.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:08.184039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:08.185270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:08.187742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 483/954] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:07 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:08.194811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d321af-30b3-4c8e-9d42-559064991a99 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.242041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b9534f8-e91d-4314-9f6f-94977cc397de None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.266515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.284837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b58e4-7a8e-458e-967b-e2906084e79b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.285932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b58e4-7a8e-458e-967b-e2906084e79b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.286395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b58e4-7a8e-458e-967b-e2906084e79b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.286818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d27b58e4-7a8e-458e-967b-e2906084e79b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.287632 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b58e4-7a8e-458e-967b-e2906084e79b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.289156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d27b58e4-7a8e-458e-967b-e2906084e79b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-d27b58e4-7a8e-458e-967b-e2906084e79b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:08.302352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:08.303743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:08.304779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:08.312298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 472/955] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:08.313432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b58e4-7a8e-458e-967b-e2906084e79b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.331581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d321af-30b3-4c8e-9d42-559064991a99 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.375323 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ac066-ec0b-4c50-9208-bbee1f75735f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.376243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ac066-ec0b-4c50-9208-bbee1f75735f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.376696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ac066-ec0b-4c50-9208-bbee1f75735f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.377126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2ac066-ec0b-4c50-9208-bbee1f75735f None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.378061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ac066-ec0b-4c50-9208-bbee1f75735f None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:08.378561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ac066-ec0b-4c50-9208-bbee1f75735f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.380118 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2ac066-ec0b-4c50-9208-bbee1f75735f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.389981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 484/956] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:08.395887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ac066-ec0b-4c50-9208-bbee1f75735f None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.409376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.444939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d27b58e4-7a8e-458e-967b-e2906084e79b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.483661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-501db65b-0545-4cd8-b5d5-8a70be5b116a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.484334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db65b-0545-4cd8-b5d5-8a70be5b116a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.484718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db65b-0545-4cd8-b5d5-8a70be5b116a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.485014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-501db65b-0545-4cd8-b5d5-8a70be5b116a None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.485751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501db65b-0545-4cd8-b5d5-8a70be5b116a None admin] RBAC: Adding query filter params (name=compute) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:08.486066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501db65b-0545-4cd8-b5d5-8a70be5b116a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.487574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-501db65b-0545-4cd8-b5d5-8a70be5b116a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.493362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 473/957] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:08.495216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c424e9c-0531-4d1a-b5e1-0b4fa5f38c11 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.495523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c424e9c-0531-4d1a-b5e1-0b4fa5f38c11 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.495805 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c424e9c-0531-4d1a-b5e1-0b4fa5f38c11 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.497195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 474/958] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:08.501948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c424e9c-0531-4d1a-b5e1-0b4fa5f38c11 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.514582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2ac066-ec0b-4c50-9208-bbee1f75735f None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.532245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.556510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f21c6b-550d-4c19-a7c7-de4a2da4a703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.558171 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f21c6b-550d-4c19-a7c7-de4a2da4a703 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.558490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f21c6b-550d-4c19-a7c7-de4a2da4a703 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.559131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f21c6b-550d-4c19-a7c7-de4a2da4a703 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.571594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f21c6b-550d-4c19-a7c7-de4a2da4a703 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.574792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f21c6b-550d-4c19-a7c7-de4a2da4a703 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-64f21c6b-550d-4c19-a7c7-de4a2da4a703 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:08.594158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:08.595185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:08.596089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:08.596089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:08.596089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:08.596089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:08.600358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 485/959] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:08.601422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22999a57-0842-46de-bc61-a7059a2a9ba9 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.601741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22999a57-0842-46de-bc61-a7059a2a9ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.602023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22999a57-0842-46de-bc61-a7059a2a9ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.611387 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c424e9c-0531-4d1a-b5e1-0b4fa5f38c11 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.618297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-22999a57-0842-46de-bc61-a7059a2a9ba9 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:08.620040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-22999a57-0842-46de-bc61-a7059a2a9ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:08.649372 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8e442-db67-49d3-af32-53b7f9142ec1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.650090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8e442-db67-49d3-af32-53b7f9142ec1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.650376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8e442-db67-49d3-af32-53b7f9142ec1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.650742 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c8e442-db67-49d3-af32-53b7f9142ec1 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.651450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8e442-db67-49d3-af32-53b7f9142ec1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:08.651759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8e442-db67-49d3-af32-53b7f9142ec1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.653141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c8e442-db67-49d3-af32-53b7f9142ec1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.654662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.669539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 475/960] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:08.669539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8e442-db67-49d3-af32-53b7f9142ec1 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.714592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 486/961] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:08.732697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22999a57-0842-46de-bc61-a7059a2a9ba9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.800265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c8e442-db67-49d3-af32-53b7f9142ec1 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.853302 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:08.858792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783e4a4-f29d-4a69-9e85-9c09e4b2ff42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.860051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783e4a4-f29d-4a69-9e85-9c09e4b2ff42 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.860051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783e4a4-f29d-4a69-9e85-9c09e4b2ff42 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.860051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d783e4a4-f29d-4a69-9e85-9c09e4b2ff42 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.860732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783e4a4-f29d-4a69-9e85-9c09e4b2ff42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:08.861039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783e4a4-f29d-4a69-9e85-9c09e4b2ff42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.862457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d783e4a4-f29d-4a69-9e85-9c09e4b2ff42 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.865259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22999a57-0842-46de-bc61-a7059a2a9ba9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:08.877386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 476/962] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:08.883226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783e4a4-f29d-4a69-9e85-9c09e4b2ff42 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.918816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf0bba-c5ad-4d05-a404-a461a0e35e99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:08.919544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf0bba-c5ad-4d05-a404-a461a0e35e99 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:08.919828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf0bba-c5ad-4d05-a404-a461a0e35e99 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:08.921292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbf0bba-c5ad-4d05-a404-a461a0e35e99 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:08.922027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf0bba-c5ad-4d05-a404-a461a0e35e99 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:08.923409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbf0bba-c5ad-4d05-a404-a461a0e35e99 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-bbbf0bba-c5ad-4d05-a404-a461a0e35e99 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:08.935599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:08.936658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:08.937489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 487/963] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:08.943512 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf0bba-c5ad-4d05-a404-a461a0e35e99 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:08.958920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.007943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d783e4a4-f29d-4a69-9e85-9c09e4b2ff42 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.046963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3f3d9-ba7c-4a6f-8aa4-f35dfbcb6b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.047924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3f3d9-ba7c-4a6f-8aa4-f35dfbcb6b1e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.048274 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3f3d9-ba7c-4a6f-8aa4-f35dfbcb6b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.048620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b3f3d9-ba7c-4a6f-8aa4-f35dfbcb6b1e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.058872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3f3d9-ba7c-4a6f-8aa4-f35dfbcb6b1e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.060659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b3f3d9-ba7c-4a6f-8aa4-f35dfbcb6b1e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.071116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbf0bba-c5ad-4d05-a404-a461a0e35e99 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-09b3f3d9-ba7c-4a6f-8aa4-f35dfbcb6b1e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:09.077441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:09.078431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:09.079377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:09.079377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:09.079377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:09.079377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:09.079377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:09.079377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:09.079377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:09.079377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:09.079377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 20 19:11:09.079377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:09.083790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 477/964] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:09.084721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3f3d9-ba7c-4a6f-8aa4-f35dfbcb6b1e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.111597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643c157-5e94-4bc6-aaf3-7d6d7fb8eb8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.112277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643c157-5e94-4bc6-aaf3-7d6d7fb8eb8c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.112727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643c157-5e94-4bc6-aaf3-7d6d7fb8eb8c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.113010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b643c157-5e94-4bc6-aaf3-7d6d7fb8eb8c None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.114077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643c157-5e94-4bc6-aaf3-7d6d7fb8eb8c None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:09.114534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643c157-5e94-4bc6-aaf3-7d6d7fb8eb8c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.116331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b643c157-5e94-4bc6-aaf3-7d6d7fb8eb8c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.125590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 488/965] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:09.129545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643c157-5e94-4bc6-aaf3-7d6d7fb8eb8c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.170646 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.213421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b3f3d9-ba7c-4a6f-8aa4-f35dfbcb6b1e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.232817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.252599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f6957-7f2d-4aff-a37d-f57bda1cb0cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.253631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f6957-7f2d-4aff-a37d-f57bda1cb0cb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.254118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f6957-7f2d-4aff-a37d-f57bda1cb0cb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.254586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60f6957-7f2d-4aff-a37d-f57bda1cb0cb None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.255596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f6957-7f2d-4aff-a37d-f57bda1cb0cb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:09.256099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f6957-7f2d-4aff-a37d-f57bda1cb0cb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.257783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60f6957-7f2d-4aff-a37d-f57bda1cb0cb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.271613 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b643c157-5e94-4bc6-aaf3-7d6d7fb8eb8c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.284736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 478/966] 10.222.0.69 () {56 vars in 1149 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:09.286624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f6957-7f2d-4aff-a37d-f57bda1cb0cb None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.316088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1eac27-5e89-42ba-8a93-c313d2abc506 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.316088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1eac27-5e89-42ba-8a93-c313d2abc506 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.316088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1eac27-5e89-42ba-8a93-c313d2abc506 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.316088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1eac27-5e89-42ba-8a93-c313d2abc506 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.316519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1eac27-5e89-42ba-8a93-c313d2abc506 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:09.316590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1eac27-5e89-42ba-8a93-c313d2abc506 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.318282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1eac27-5e89-42ba-8a93-c313d2abc506 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.329583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 489/967] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:09.335224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1eac27-5e89-42ba-8a93-c313d2abc506 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.417332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60f6957-7f2d-4aff-a37d-f57bda1cb0cb None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.452154 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.468153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddfb96a1-5d59-4597-807b-a733a6cb09dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.469178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb96a1-5d59-4597-807b-a733a6cb09dd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.469714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb96a1-5d59-4597-807b-a733a6cb09dd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.469824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1eac27-5e89-42ba-8a93-c313d2abc506 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.470301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddfb96a1-5d59-4597-807b-a733a6cb09dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.514595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3c37c-f259-4b1e-bc7b-0f1a8bec9514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.515263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3c37c-f259-4b1e-bc7b-0f1a8bec9514 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.515685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3c37c-f259-4b1e-bc7b-0f1a8bec9514 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.516079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c3c37c-f259-4b1e-bc7b-0f1a8bec9514 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.516831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3c37c-f259-4b1e-bc7b-0f1a8bec9514 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.518259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c3c37c-f259-4b1e-bc7b-0f1a8bec9514 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.535080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.553562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 490/968] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:09.572377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb96a1-5d59-4597-807b-a733a6cb09dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.574957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddfb96a1-5d59-4597-807b-a733a6cb09dd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.695719 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 479/969] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:09.702735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3c37c-f259-4b1e-bc7b-0f1a8bec9514 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.778519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7846ce7-bc7a-4491-b7d9-9fddf3620ab4 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.778846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7846ce7-bc7a-4491-b7d9-9fddf3620ab4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.779093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7846ce7-bc7a-4491-b7d9-9fddf3620ab4 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.780723 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 480/970] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:09.786027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ea460-0346-48e5-b47a-2d18809f44ed None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.786319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ea460-0346-48e5-b47a-2d18809f44ed None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.786601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945ea460-0346-48e5-b47a-2d18809f44ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.806335 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-945ea460-0346-48e5-b47a-2d18809f44ed None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:09.808560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-945ea460-0346-48e5-b47a-2d18809f44ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:09.830704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c3c37c-f259-4b1e-bc7b-0f1a8bec9514 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:09.876025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7def5-5874-4a5b-8016-5cf09f77de7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:09.876841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7def5-5874-4a5b-8016-5cf09f77de7d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:09.877117 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7def5-5874-4a5b-8016-5cf09f77de7d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:09.877402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e7def5-5874-4a5b-8016-5cf09f77de7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:09.887574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:09.890330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7def5-5874-4a5b-8016-5cf09f77de7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:09.891914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84e7def5-5874-4a5b-8016-5cf09f77de7d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:09.905622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 481/971] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-84e7def5-5874-4a5b-8016-5cf09f77de7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:09.920461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:09.921398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:09.922698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:09.922698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:09.922698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:09.922698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:09.922698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 491/972] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:09.927258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ea460-0346-48e5-b47a-2d18809f44ed None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.939880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7def5-5874-4a5b-8016-5cf09f77de7d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:09.943542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.041957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945ea460-0346-48e5-b47a-2d18809f44ed None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.055265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e7def5-5874-4a5b-8016-5cf09f77de7d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.078720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f3206-f81d-492d-90b9-3d2b02e7d69f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.079520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3206-f81d-492d-90b9-3d2b02e7d69f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.079811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3206-f81d-492d-90b9-3d2b02e7d69f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.080095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1f3206-f81d-492d-90b9-3d2b02e7d69f None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.080824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f3206-f81d-492d-90b9-3d2b02e7d69f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:10.081136 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f3206-f81d-492d-90b9-3d2b02e7d69f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.082551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1f3206-f81d-492d-90b9-3d2b02e7d69f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.095422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.100609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 482/973] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:10.100763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.100763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.100763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.100763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.101446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.102806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WARNING keystone.server.flask.common [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 20 19:11:10.105329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f3206-f81d-492d-90b9-3d2b02e7d69f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.120834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.sql.core [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 19:11:10.120834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 19:11:10.120834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [parameters: {'user_id': 'f2a7a85402084115b5780e5f191708ca', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 19:11:10.120834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80210) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application result.read() Jul 20 19:11:10.142824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:11:10.143798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application yield resource Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application next(self.gen) Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self.session.rollback() Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self.force_reraise() Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise self.value Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application session.commit() Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 20 19:11:10.149815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._prepare_impl() Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self.session.flush() Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._flush(objects) Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application compat.raise_( Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application flush_context.execute() Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rec.execute(self) Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 20 19:11:10.151476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 20 19:11:10.152450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application result = self._query(query) Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application conn.query(q) Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application result.read() Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 20 19:11:10.153437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [parameters: {'user_id': 'f2a7a85402084115b5780e5f191708ca', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:10.156291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 20 19:11:10.158341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 20 19:11:10.160570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 20 19:11:10.160570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 20 19:11:10.160570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 20 19:11:10.160570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 20 19:11:10.160570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 20 19:11:10.160570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:10.160570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 492/974] 10.222.0.69 () {60 vars in 1131 bytes} [Wed Jul 20 19:11:09 2022] POST /identity/v3/users => generated 160 bytes in 205 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:10.160570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.192967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.228267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.230894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1f3206-f81d-492d-90b9-3d2b02e7d69f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.277356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a30d6-2be2-449f-b89e-435ca1ea6192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.278660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a30d6-2be2-449f-b89e-435ca1ea6192 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.279342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a30d6-2be2-449f-b89e-435ca1ea6192 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.279781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314a30d6-2be2-449f-b89e-435ca1ea6192 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.286739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17814f62-84a8-4f6f-9108-ef8daf543b4e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.288700 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a30d6-2be2-449f-b89e-435ca1ea6192 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.291880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314a30d6-2be2-449f-b89e-435ca1ea6192 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-314a30d6-2be2-449f-b89e-435ca1ea6192 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:10.321070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:10.322115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:10.324068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:10.324068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:10.324068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:10.324068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:10.324068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:10.324068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:10.324068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:10.324068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:10.326603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 483/975] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:10.331895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a30d6-2be2-449f-b89e-435ca1ea6192 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.339033 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac439c5-5a85-451c-b28b-e9c5cce5b0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.339920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac439c5-5a85-451c-b28b-e9c5cce5b0ae None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.340276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac439c5-5a85-451c-b28b-e9c5cce5b0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.340620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac439c5-5a85-451c-b28b-e9c5cce5b0ae None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.350870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac439c5-5a85-451c-b28b-e9c5cce5b0ae None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.353368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac439c5-5a85-451c-b28b-e9c5cce5b0ae None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-bac439c5-5a85-451c-b28b-e9c5cce5b0ae None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:10.374594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:10.375700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:10.376657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:10.376657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:10.376657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:10.376657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:10.376657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:10.376657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:10.376657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:10.376657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:10.376657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 20 19:11:10.376657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:10.381694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 493/976] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:10.382624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac439c5-5a85-451c-b28b-e9c5cce5b0ae None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.477428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314a30d6-2be2-449f-b89e-435ca1ea6192 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.505437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac439c5-5a85-451c-b28b-e9c5cce5b0ae None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.516168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.526290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99a02a-da54-42bf-a35a-e0a8b57e49dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.527051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99a02a-da54-42bf-a35a-e0a8b57e49dd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.527337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99a02a-da54-42bf-a35a-e0a8b57e49dd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.527615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d99a02a-da54-42bf-a35a-e0a8b57e49dd None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.528408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99a02a-da54-42bf-a35a-e0a8b57e49dd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:10.528718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99a02a-da54-42bf-a35a-e0a8b57e49dd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.530184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d99a02a-da54-42bf-a35a-e0a8b57e49dd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.530410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.549404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 484/977] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:10.550337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99a02a-da54-42bf-a35a-e0a8b57e49dd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:10.557450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba2a1d0-9b5a-48b0-b7d0-eee5d9a344f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.558491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2a1d0-9b5a-48b0-b7d0-eee5d9a344f8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.558972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2a1d0-9b5a-48b0-b7d0-eee5d9a344f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.559502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba2a1d0-9b5a-48b0-b7d0-eee5d9a344f8 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.561082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2a1d0-9b5a-48b0-b7d0-eee5d9a344f8 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:10.561635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2a1d0-9b5a-48b0-b7d0-eee5d9a344f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.563727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ba2a1d0-9b5a-48b0-b7d0-eee5d9a344f8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.617720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 494/978] 10.222.0.69 () {56 vars in 1101 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:10.669357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d99a02a-da54-42bf-a35a-e0a8b57e49dd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:10.711074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd6ef9f-3db1-493e-aba4-3e63d35bd34a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:10.712105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6ef9f-3db1-493e-aba4-3e63d35bd34a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.712588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6ef9f-3db1-493e-aba4-3e63d35bd34a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.713615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd6ef9f-3db1-493e-aba4-3e63d35bd34a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.714732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6ef9f-3db1-493e-aba4-3e63d35bd34a None admin] RBAC: Adding query filter params (role.id=3e0a48dd676e4ee0b8f3cad1dbe68a6b, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=cbd4016436b54a0fb8c74c35273957a0) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:10.715288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6ef9f-3db1-493e-aba4-3e63d35bd34a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:10.716982 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd6ef9f-3db1-493e-aba4-3e63d35bd34a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:10.724758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 485/979] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:10 2022] GET /identity/v3/role_assignments?user.id=cbd4016436b54a0fb8c74c35273957a0&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=3e0a48dd676e4ee0b8f3cad1dbe68a6b => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:10.838917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf19899-74f3-43cc-b1ed-018b23057577 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.839253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf19899-74f3-43cc-b1ed-018b23057577 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.839479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf19899-74f3-43cc-b1ed-018b23057577 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.841286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 495/980] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:10.847128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a7024-8557-4947-aa50-f22511060b4c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:10.847421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a7024-8557-4947-aa50-f22511060b4c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:10.847711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0a7024-8557-4947-aa50-f22511060b4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:10.867321 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-bd0a7024-8557-4947-aa50-f22511060b4c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:10.868817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bd0a7024-8557-4947-aa50-f22511060b4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:10.924799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:10.949129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 486/981] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:10.974068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf19899-74f3-43cc-b1ed-018b23057577 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:11.079927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf19899-74f3-43cc-b1ed-018b23057577 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:11.118007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf786abf-eb12-4b39-9ef1-f5a8ad794a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:11.118663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf786abf-eb12-4b39-9ef1-f5a8ad794a29 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:11.119063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf786abf-eb12-4b39-9ef1-f5a8ad794a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:11.119416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf786abf-eb12-4b39-9ef1-f5a8ad794a29 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:11.120757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf786abf-eb12-4b39-9ef1-f5a8ad794a29 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:11.122159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf786abf-eb12-4b39-9ef1-f5a8ad794a29 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:11.138187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 496/982] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:10 2022] POST /identity/v3/services => generated 242 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:11:11.756935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcbdca-837e-4b8a-a5bd-c6a99d39b5fd None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:11.757461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcbdca-837e-4b8a-a5bd-c6a99d39b5fd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:11.757598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fcbdca-837e-4b8a-a5bd-c6a99d39b5fd None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:11.758953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 487/983] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:11.764862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e81d90-20ef-40a8-a1d1-15d0e5f4416e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:11.765160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e81d90-20ef-40a8-a1d1-15d0e5f4416e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:11.765436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e81d90-20ef-40a8-a1d1-15d0e5f4416e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:11.782365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f1e81d90-20ef-40a8-a1d1-15d0e5f4416e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:11.783874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f1e81d90-20ef-40a8-a1d1-15d0e5f4416e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:11.810038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:11.862770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 497/984] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:11 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:11.886288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcbdca-837e-4b8a-a5bd-c6a99d39b5fd None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:11.993779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fcbdca-837e-4b8a-a5bd-c6a99d39b5fd None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.027544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bcdf0-7da6-40ac-981d-8f7a1df22798 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.028280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bcdf0-7da6-40ac-981d-8f7a1df22798 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.028563 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bcdf0-7da6-40ac-981d-8f7a1df22798 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.028838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687bcdf0-7da6-40ac-981d-8f7a1df22798 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.029483 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687bcdf0-7da6-40ac-981d-8f7a1df22798 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.030876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687bcdf0-7da6-40ac-981d-8f7a1df22798 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-687bcdf0-7da6-40ac-981d-8f7a1df22798 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:12.043350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:12.044389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:12.045226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 488/985] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:12.052808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e81d90-20ef-40a8-a1d1-15d0e5f4416e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:12.155416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e81d90-20ef-40a8-a1d1-15d0e5f4416e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.190122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db40bf-bfbd-4733-a0c5-9073fbd3c85b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.190775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db40bf-bfbd-4733-a0c5-9073fbd3c85b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.191053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db40bf-bfbd-4733-a0c5-9073fbd3c85b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.191323 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97db40bf-bfbd-4733-a0c5-9073fbd3c85b None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.192013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db40bf-bfbd-4733-a0c5-9073fbd3c85b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:12.192315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db40bf-bfbd-4733-a0c5-9073fbd3c85b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.193689 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97db40bf-bfbd-4733-a0c5-9073fbd3c85b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.201588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 498/986] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:12.204594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bcdf0-7da6-40ac-981d-8f7a1df22798 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:12.253838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:12.313766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687bcdf0-7da6-40ac-981d-8f7a1df22798 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.353555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0929c7a-b12a-4529-b487-20279bcea0a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.354741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0929c7a-b12a-4529-b487-20279bcea0a9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.355033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0929c7a-b12a-4529-b487-20279bcea0a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.355313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0929c7a-b12a-4529-b487-20279bcea0a9 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.356019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0929c7a-b12a-4529-b487-20279bcea0a9 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:12.356330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0929c7a-b12a-4529-b487-20279bcea0a9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.357730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0929c7a-b12a-4529-b487-20279bcea0a9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.367575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 489/987] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:12.368766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db40bf-bfbd-4733-a0c5-9073fbd3c85b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:12.395227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:12.472569 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97db40bf-bfbd-4733-a0c5-9073fbd3c85b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.506913 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada199b-918e-497f-bb51-502e0028badc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.507568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada199b-918e-497f-bb51-502e0028badc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.507850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada199b-918e-497f-bb51-502e0028badc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.508125 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada199b-918e-497f-bb51-502e0028badc None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.508916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada199b-918e-497f-bb51-502e0028badc None admin] RBAC: Adding query filter params (interface=admin, service_id=c1d906f25dc0453d89b7b975a79137c9, region_id=RegionOne) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:12.509231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada199b-918e-497f-bb51-502e0028badc None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.511185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada199b-918e-497f-bb51-502e0028badc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.520786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 499/988] 10.222.0.69 () {56 vars in 1249 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/endpoints?service_id=c1d906f25dc0453d89b7b975a79137c9&interface=admin®ion_id=RegionOne => generated 193 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:12.521943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0929c7a-b12a-4529-b487-20279bcea0a9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:12.624173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0929c7a-b12a-4529-b487-20279bcea0a9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:12.633327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:12.660021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83cf7fba-d9f0-4664-bc27-79198d41c9d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:12.660689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf7fba-d9f0-4664-bc27-79198d41c9d5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.660972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf7fba-d9f0-4664-bc27-79198d41c9d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.661248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83cf7fba-d9f0-4664-bc27-79198d41c9d5 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.661962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf7fba-d9f0-4664-bc27-79198d41c9d5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:12.663330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83cf7fba-d9f0-4664-bc27-79198d41c9d5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:12.677051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 490/989] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:11:12.858456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875aa8d-1c85-4282-9f64-bf67fcfafa36 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.858787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875aa8d-1c85-4282-9f64-bf67fcfafa36 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.859031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e875aa8d-1c85-4282-9f64-bf67fcfafa36 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.860436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 500/990] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:12.865695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb08728-b662-4f91-9ea5-eba09b3a2bdd None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:12.866156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb08728-b662-4f91-9ea5-eba09b3a2bdd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:12.866354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb08728-b662-4f91-9ea5-eba09b3a2bdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:12.883168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7fb08728-b662-4f91-9ea5-eba09b3a2bdd None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:12.884454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7fb08728-b662-4f91-9ea5-eba09b3a2bdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:12.956036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 491/991] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:12 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:12.979852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875aa8d-1c85-4282-9f64-bf67fcfafa36 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.069547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.087410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e875aa8d-1c85-4282-9f64-bf67fcfafa36 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.124617 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871c603a-9f22-4c60-a409-9850f74a3186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.125413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c603a-9f22-4c60-a409-9850f74a3186 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.125724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c603a-9f22-4c60-a409-9850f74a3186 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.126006 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871c603a-9f22-4c60-a409-9850f74a3186 None admin] PATH_INFO: `/v3/services/image` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.126725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c603a-9f22-4c60-a409-9850f74a3186 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.128188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871c603a-9f22-4c60-a409-9850f74a3186 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-871c603a-9f22-4c60-a409-9850f74a3186 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:13.141120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:13.142389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:13.143250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:13.144615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 501/992] 10.222.0.69 () {56 vars in 1105 bytes} [Wed Jul 20 19:11:12 2022] GET /identity/v3/services/image => generated 86 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:13.148239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb08728-b662-4f91-9ea5-eba09b3a2bdd None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.225364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167e4e6-afe7-4b0e-a3a0-3748c8455f9e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.225919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167e4e6-afe7-4b0e-a3a0-3748c8455f9e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.226001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f167e4e6-afe7-4b0e-a3a0-3748c8455f9e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.227537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 502/993] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:13.232684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560964af-929e-47a5-98f1-265138c99ce8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.232970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560964af-929e-47a5-98f1-265138c99ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.233245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-560964af-929e-47a5-98f1-265138c99ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.250871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-560964af-929e-47a5-98f1-265138c99ce8 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:13.252060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-560964af-929e-47a5-98f1-265138c99ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:13.255270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb08728-b662-4f91-9ea5-eba09b3a2bdd None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.256815 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.292675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb1ef9-b3c6-4486-ac32-5f5415bc9c7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.293633 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb1ef9-b3c6-4486-ac32-5f5415bc9c7f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.294065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb1ef9-b3c6-4486-ac32-5f5415bc9c7f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.294681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eb1ef9-b3c6-4486-ac32-5f5415bc9c7f None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.295553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb1ef9-b3c6-4486-ac32-5f5415bc9c7f None admin] RBAC: Adding query filter params (name=image) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:13.296012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb1ef9-b3c6-4486-ac32-5f5415bc9c7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.297591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eb1ef9-b3c6-4486-ac32-5f5415bc9c7f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.303040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 492/994] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/services?name=image => generated 122 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:13.309398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb1ef9-b3c6-4486-ac32-5f5415bc9c7f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.337111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 503/995] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:13.360846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560964af-929e-47a5-98f1-265138c99ce8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.419703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eb1ef9-b3c6-4486-ac32-5f5415bc9c7f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.446176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.457319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eb5c9-01f1-4e4e-83ec-af5adff5d2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.458025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eb5c9-01f1-4e4e-83ec-af5adff5d2ee None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.458314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eb5c9-01f1-4e4e-83ec-af5adff5d2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.458600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220eb5c9-01f1-4e4e-83ec-af5adff5d2ee None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.459310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220eb5c9-01f1-4e4e-83ec-af5adff5d2ee None admin] RBAC: Adding query filter params (type=image) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:13.459608 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220eb5c9-01f1-4e4e-83ec-af5adff5d2ee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.460981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220eb5c9-01f1-4e4e-83ec-af5adff5d2ee None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.466790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 493/996] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:13.472859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-560964af-929e-47a5-98f1-265138c99ce8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.513139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d229d-a982-4116-874e-7fd3e6c63efe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.513993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d229d-a982-4116-874e-7fd3e6c63efe None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.514279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d229d-a982-4116-874e-7fd3e6c63efe None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.514557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1d229d-a982-4116-874e-7fd3e6c63efe None admin] PATH_INFO: `/v3/services/compute` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.515234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d229d-a982-4116-874e-7fd3e6c63efe None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.516649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1d229d-a982-4116-874e-7fd3e6c63efe None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-cf1d229d-a982-4116-874e-7fd3e6c63efe None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:13.529703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:13.530827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:13.531730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:13.533186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 504/997] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/services/compute => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:13.537382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eb5c9-01f1-4e4e-83ec-af5adff5d2ee None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.572399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.650966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220eb5c9-01f1-4e4e-83ec-af5adff5d2ee None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.685965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ba78-2e56-40c1-8b89-b7898bbd1445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.686556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ba78-2e56-40c1-8b89-b7898bbd1445 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.686837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ba78-2e56-40c1-8b89-b7898bbd1445 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.687110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ba78-2e56-40c1-8b89-b7898bbd1445 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.687806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8ba78-2e56-40c1-8b89-b7898bbd1445 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:13.688110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8ba78-2e56-40c1-8b89-b7898bbd1445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.689469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8ba78-2e56-40c1-8b89-b7898bbd1445 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.694491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 494/998] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:13.700138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d229d-a982-4116-874e-7fd3e6c63efe None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.805360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1d229d-a982-4116-874e-7fd3e6c63efe None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.841164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb1059-ebbc-45a7-9082-3554571334b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.841851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb1059-ebbc-45a7-9082-3554571334b9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.842135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb1059-ebbc-45a7-9082-3554571334b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.842409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fbb1059-ebbc-45a7-9082-3554571334b9 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.843107 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb1059-ebbc-45a7-9082-3554571334b9 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:13.843413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb1059-ebbc-45a7-9082-3554571334b9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:13.844769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fbb1059-ebbc-45a7-9082-3554571334b9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:13.854721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 505/999] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:13.855925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ba78-2e56-40c1-8b89-b7898bbd1445 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:13.953504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:13.962082 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ba78-2e56-40c1-8b89-b7898bbd1445 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:13.996972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4625f5b-4215-40ad-aba4-7391d0131dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:13.997660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4625f5b-4215-40ad-aba4-7391d0131dac None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:13.997943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4625f5b-4215-40ad-aba4-7391d0131dac None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:13.998218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4625f5b-4215-40ad-aba4-7391d0131dac None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:13.999001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4625f5b-4215-40ad-aba4-7391d0131dac None admin] RBAC: Adding query filter params (interface=public, service_id=5cfe7aa1008c4cac9a1301de1484a76d, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:13.999312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4625f5b-4215-40ad-aba4-7391d0131dac None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.000679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4625f5b-4215-40ad-aba4-7391d0131dac None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.010792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 495/1000] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:13 2022] GET /identity/v3/endpoints?service_id=5cfe7aa1008c4cac9a1301de1484a76d&interface=public®ion_id=RegionOne => generated 194 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:14.011847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb1059-ebbc-45a7-9082-3554571334b9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.022675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:14.113744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fbb1059-ebbc-45a7-9082-3554571334b9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.148979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70805bc6-2563-482b-bf11-7db10314dca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.149673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70805bc6-2563-482b-bf11-7db10314dca1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.149979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70805bc6-2563-482b-bf11-7db10314dca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.150224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70805bc6-2563-482b-bf11-7db10314dca1 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.150951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70805bc6-2563-482b-bf11-7db10314dca1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.152309 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70805bc6-2563-482b-bf11-7db10314dca1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:14.165551 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 506/1001] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:11:14.723361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286d6d8-c8aa-4b4a-9163-3785db76b92d None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.723744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286d6d8-c8aa-4b4a-9163-3785db76b92d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.723925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c286d6d8-c8aa-4b4a-9163-3785db76b92d None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.725332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 496/1002] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:14.730640 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28411-0ef1-4bb6-9d86-cb365bd77236 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.731077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28411-0ef1-4bb6-9d86-cb365bd77236 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.731237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf28411-0ef1-4bb6-9d86-cb365bd77236 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.748102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-cdf28411-0ef1-4bb6-9d86-cb365bd77236 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:14.749275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cdf28411-0ef1-4bb6-9d86-cb365bd77236 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:14.815997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 507/1003] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:14.838917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286d6d8-c8aa-4b4a-9163-3785db76b92d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:14.943253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c286d6d8-c8aa-4b4a-9163-3785db76b92d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:14.985633 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5987e1c-4caa-4d84-aa8d-fb781281f914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:14.986383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5987e1c-4caa-4d84-aa8d-fb781281f914 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:14.986641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5987e1c-4caa-4d84-aa8d-fb781281f914 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:14.986914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5987e1c-4caa-4d84-aa8d-fb781281f914 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:14.987563 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5987e1c-4caa-4d84-aa8d-fb781281f914 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:14.988929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5987e1c-4caa-4d84-aa8d-fb781281f914 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e5987e1c-4caa-4d84-aa8d-fb781281f914 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:15.001889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:15.002956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 20 19:11:15.003845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:15.005829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 497/1004] 10.222.0.69 () {56 vars in 1126 bytes} [Wed Jul 20 19:11:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:15.009163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28411-0ef1-4bb6-9d86-cb365bd77236 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:15.011626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:15.112142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf28411-0ef1-4bb6-9d86-cb365bd77236 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:15.148051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b0053-3e27-4fda-bfe8-031a1f929bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:15.148714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b0053-3e27-4fda-bfe8-031a1f929bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.148995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b0053-3e27-4fda-bfe8-031a1f929bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.149269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388b0053-3e27-4fda-bfe8-031a1f929bf6 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.150098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b0053-3e27-4fda-bfe8-031a1f929bf6 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:15.150413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b0053-3e27-4fda-bfe8-031a1f929bf6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:15.151957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388b0053-3e27-4fda-bfe8-031a1f929bf6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:15.161721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 508/1005] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:15.163217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5987e1c-4caa-4d84-aa8d-fb781281f914 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:15.190022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:15.268614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5987e1c-4caa-4d84-aa8d-fb781281f914 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:15.306284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7c80e9d-dc63-4834-98fa-b6a9b358f304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:15.307560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c80e9d-dc63-4834-98fa-b6a9b358f304 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.307956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c80e9d-dc63-4834-98fa-b6a9b358f304 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.308237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c80e9d-dc63-4834-98fa-b6a9b358f304 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.308943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c80e9d-dc63-4834-98fa-b6a9b358f304 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:15.309251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c80e9d-dc63-4834-98fa-b6a9b358f304 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:15.310652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7c80e9d-dc63-4834-98fa-b6a9b358f304 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:15.324553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 498/1006] 10.222.0.69 () {56 vars in 1122 bytes} [Wed Jul 20 19:11:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:15.330103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b0053-3e27-4fda-bfe8-031a1f929bf6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:15.396691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:15.432903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388b0053-3e27-4fda-bfe8-031a1f929bf6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:15.467716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c55799-bf66-41ac-9368-b7231f0ba035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:15.468376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55799-bf66-41ac-9368-b7231f0ba035 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.468658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55799-bf66-41ac-9368-b7231f0ba035 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.468934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c55799-bf66-41ac-9368-b7231f0ba035 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.469786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c55799-bf66-41ac-9368-b7231f0ba035 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:15.471151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c55799-bf66-41ac-9368-b7231f0ba035 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:15.493598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 509/1007] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:15.536381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3be905-e06e-4272-bbc1-61a92e8af044 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.536577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3be905-e06e-4272-bbc1-61a92e8af044 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.536848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3be905-e06e-4272-bbc1-61a92e8af044 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.538491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 499/1008] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:15.544802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81476ffd-a4e9-4e91-a826-58d7e284af9e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.545231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81476ffd-a4e9-4e91-a826-58d7e284af9e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.545644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81476ffd-a4e9-4e91-a826-58d7e284af9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.564282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-81476ffd-a4e9-4e91-a826-58d7e284af9e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:15.565490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-81476ffd-a4e9-4e91-a826-58d7e284af9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:15.651760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 510/1009] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:15 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:15.675570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3be905-e06e-4272-bbc1-61a92e8af044 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:15.769775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:15.787860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3be905-e06e-4272-bbc1-61a92e8af044 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:15.824686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2de9b25-15a8-46ab-b78b-2fa93794daf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:15.825364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de9b25-15a8-46ab-b78b-2fa93794daf8 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:15.825673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de9b25-15a8-46ab-b78b-2fa93794daf8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:15.825944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2de9b25-15a8-46ab-b78b-2fa93794daf8 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:15.826660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de9b25-15a8-46ab-b78b-2fa93794daf8 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:15.828024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2de9b25-15a8-46ab-b78b-2fa93794daf8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:15.843524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 500/1010] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:15 2022] POST /identity/v3/services => generated 242 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:16.224030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f780307-58bf-4897-a7dd-55e039719a93 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.224553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f780307-58bf-4897-a7dd-55e039719a93 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.224553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f780307-58bf-4897-a7dd-55e039719a93 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.226114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 511/1011] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:16.231750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9c319-8b12-4f9f-83b1-dcf194d39b20 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.232205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9c319-8b12-4f9f-83b1-dcf194d39b20 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.232289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63c9c319-8b12-4f9f-83b1-dcf194d39b20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.249279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-63c9c319-8b12-4f9f-83b1-dcf194d39b20 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:16.250502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-63c9c319-8b12-4f9f-83b1-dcf194d39b20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:16.342602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 501/1012] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:16 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:16.366199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f780307-58bf-4897-a7dd-55e039719a93 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.423940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:16.474873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f780307-58bf-4897-a7dd-55e039719a93 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:16.510624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ecb8e-afe4-4664-bd88-e81bb3526202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:16.511419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ecb8e-afe4-4664-bd88-e81bb3526202 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.511700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ecb8e-afe4-4664-bd88-e81bb3526202 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.511976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ecb8e-afe4-4664-bd88-e81bb3526202 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.512623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ecb8e-afe4-4664-bd88-e81bb3526202 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:16.514190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-206ecb8e-afe4-4664-bd88-e81bb3526202 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-206ecb8e-afe4-4664-bd88-e81bb3526202 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:16.526211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:16.527263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 20 19:11:16.528133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:16.529857 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 512/1013] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:16 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:16.533265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9c319-8b12-4f9f-83b1-dcf194d39b20 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.605439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:16.641117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63c9c319-8b12-4f9f-83b1-dcf194d39b20 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:16.677427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ef6e4-82f5-4a7b-b2be-75f1acb56911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:16.678124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ef6e4-82f5-4a7b-b2be-75f1acb56911 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.678405 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ef6e4-82f5-4a7b-b2be-75f1acb56911 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.678678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66ef6e4-82f5-4a7b-b2be-75f1acb56911 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.679374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ef6e4-82f5-4a7b-b2be-75f1acb56911 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:16.679676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ef6e4-82f5-4a7b-b2be-75f1acb56911 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:16.681041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66ef6e4-82f5-4a7b-b2be-75f1acb56911 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:16.692104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 502/1014] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:16 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:16.693878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ecb8e-afe4-4664-bd88-e81bb3526202 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.806367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ecb8e-afe4-4664-bd88-e81bb3526202 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:16.822974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:16.842272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbcf5aa6-dee6-42dd-bb20-6be3d1026c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:16.842935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf5aa6-dee6-42dd-bb20-6be3d1026c24 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:16.843219 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf5aa6-dee6-42dd-bb20-6be3d1026c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:16.843494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbcf5aa6-dee6-42dd-bb20-6be3d1026c24 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:16.844258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf5aa6-dee6-42dd-bb20-6be3d1026c24 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:16.844567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf5aa6-dee6-42dd-bb20-6be3d1026c24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:16.846030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbcf5aa6-dee6-42dd-bb20-6be3d1026c24 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:16.856733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 513/1015] 10.222.0.69 () {56 vars in 1112 bytes} [Wed Jul 20 19:11:16 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:16.858011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ef6e4-82f5-4a7b-b2be-75f1acb56911 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:16.959366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66ef6e4-82f5-4a7b-b2be-75f1acb56911 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.002494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25fc81e6-982c-4cee-a6be-c6e845bd829e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.003164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc81e6-982c-4cee-a6be-c6e845bd829e None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.003448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc81e6-982c-4cee-a6be-c6e845bd829e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.003722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25fc81e6-982c-4cee-a6be-c6e845bd829e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.004460 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc81e6-982c-4cee-a6be-c6e845bd829e None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.005870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25fc81e6-982c-4cee-a6be-c6e845bd829e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.010276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:17.033541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 503/1016] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:16 2022] POST /identity/v3/endpoints => generated 334 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:17.591648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142c1ea-ae33-498b-8092-2a8b26e66a4f None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.592142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142c1ea-ae33-498b-8092-2a8b26e66a4f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.592142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1142c1ea-ae33-498b-8092-2a8b26e66a4f None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.593727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 514/1017] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:17.599290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a18b4-f37c-4179-a1b1-bf361f283e70 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.599743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a18b4-f37c-4179-a1b1-bf361f283e70 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.599926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-775a18b4-f37c-4179-a1b1-bf361f283e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.617027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-775a18b4-f37c-4179-a1b1-bf361f283e70 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:17.618247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-775a18b4-f37c-4179-a1b1-bf361f283e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:17.695224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 504/1018] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:17.718932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142c1ea-ae33-498b-8092-2a8b26e66a4f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:17.826113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1142c1ea-ae33-498b-8092-2a8b26e66a4f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:17.861181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17920139-b1dc-4c7d-931e-185bf3bdad2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:17.861931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17920139-b1dc-4c7d-931e-185bf3bdad2e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.862218 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17920139-b1dc-4c7d-931e-185bf3bdad2e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.862503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17920139-b1dc-4c7d-931e-185bf3bdad2e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.863682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17920139-b1dc-4c7d-931e-185bf3bdad2e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:17.865146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17920139-b1dc-4c7d-931e-185bf3bdad2e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:17.872227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 515/1019] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:17.873450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a18b4-f37c-4179-a1b1-bf361f283e70 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:17.908025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9325b-3dd6-41fd-a6d1-56de6d03fc21 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.908307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9325b-3dd6-41fd-a6d1-56de6d03fc21 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.908585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee9325b-3dd6-41fd-a6d1-56de6d03fc21 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.909990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 516/1020] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:17.914766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f26833-5570-4435-97e9-2c23c309b661 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:17.915057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f26833-5570-4435-97e9-2c23c309b661 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:17.915364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f26833-5570-4435-97e9-2c23c309b661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:17.933700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-34f26833-5570-4435-97e9-2c23c309b661 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:17.934891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-34f26833-5570-4435-97e9-2c23c309b661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:17.983144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:17.987581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-775a18b4-f37c-4179-a1b1-bf361f283e70 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.006535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:18.009608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 517/1021] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:18.024725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb386f-9e12-49c3-a99f-b533bd1d214c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.025454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb386f-9e12-49c3-a99f-b533bd1d214c None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.025807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb386f-9e12-49c3-a99f-b533bd1d214c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.026085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fb386f-9e12-49c3-a99f-b533bd1d214c None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.026791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb386f-9e12-49c3-a99f-b533bd1d214c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.028352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fb386f-9e12-49c3-a99f-b533bd1d214c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.033591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f26833-5570-4435-97e9-2c23c309b661 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.065543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 505/1022] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:17 2022] POST /identity/v3/projects => generated 311 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:18.158859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f26833-5570-4435-97e9-2c23c309b661 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.196809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ba3b4-6e69-4a02-84d7-90e5ea8e6122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.198287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ba3b4-6e69-4a02-84d7-90e5ea8e6122 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.198570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ba3b4-6e69-4a02-84d7-90e5ea8e6122 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.199053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a3ba3b4-6e69-4a02-84d7-90e5ea8e6122 None admin] PATH_INFO: `/v3/services/image` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.199731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ba3b4-6e69-4a02-84d7-90e5ea8e6122 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.201117 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a3ba3b4-6e69-4a02-84d7-90e5ea8e6122 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-7a3ba3b4-6e69-4a02-84d7-90e5ea8e6122 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:18.215751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:18.216816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:18.217968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 518/1023] 10.222.0.69 () {56 vars in 1105 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/services/image => generated 86 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:18.223012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb386f-9e12-49c3-a99f-b533bd1d214c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.341016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fb386f-9e12-49c3-a99f-b533bd1d214c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.376154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ce68d-ad5a-4f01-905d-b18da1c93b0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.376820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce68d-ad5a-4f01-905d-b18da1c93b0b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.377101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce68d-ad5a-4f01-905d-b18da1c93b0b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.377374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ce68d-ad5a-4f01-905d-b18da1c93b0b None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.378098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ce68d-ad5a-4f01-905d-b18da1c93b0b None admin] RBAC: Adding query filter params (name=image) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:18.378400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ce68d-ad5a-4f01-905d-b18da1c93b0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.379759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ce68d-ad5a-4f01-905d-b18da1c93b0b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.390192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 506/1024] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:18.391737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ba3b4-6e69-4a02-84d7-90e5ea8e6122 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.436367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:18.496563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a3ba3b4-6e69-4a02-84d7-90e5ea8e6122 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.529810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cf461-5ee2-4d71-bb31-d3ab5a86cf8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.530529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cf461-5ee2-4d71-bb31-d3ab5a86cf8e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.530809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cf461-5ee2-4d71-bb31-d3ab5a86cf8e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.531081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559cf461-5ee2-4d71-bb31-d3ab5a86cf8e None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.531834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cf461-5ee2-4d71-bb31-d3ab5a86cf8e None admin] RBAC: Adding query filter params (type=image) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:18.532145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cf461-5ee2-4d71-bb31-d3ab5a86cf8e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.533523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559cf461-5ee2-4d71-bb31-d3ab5a86cf8e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.543265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 519/1025] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:18.544450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ce68d-ad5a-4f01-905d-b18da1c93b0b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.580761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:18.651094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ce68d-ad5a-4f01-905d-b18da1c93b0b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.686196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddd5d10-e617-45d5-9bfc-3455ccd857a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.686915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd5d10-e617-45d5-9bfc-3455ccd857a9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.687143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd5d10-e617-45d5-9bfc-3455ccd857a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.687415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddd5d10-e617-45d5-9bfc-3455ccd857a9 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.688195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd5d10-e617-45d5-9bfc-3455ccd857a9 None admin] RBAC: Adding query filter params (interface=public, service_id=73e6e3153cb9413895f2855934cebf35, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:18.688505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd5d10-e617-45d5-9bfc-3455ccd857a9 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.689890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddd5d10-e617-45d5-9bfc-3455ccd857a9 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.699467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 507/1026] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/endpoints?service_id=73e6e3153cb9413895f2855934cebf35&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:18.700697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cf461-5ee2-4d71-bb31-d3ab5a86cf8e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:18.807211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559cf461-5ee2-4d71-bb31-d3ab5a86cf8e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:18.814815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:18.844194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3f40bf8-fa3d-4b68-9eee-6e809c09dfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:18.844858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f40bf8-fa3d-4b68-9eee-6e809c09dfee None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:18.845142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f40bf8-fa3d-4b68-9eee-6e809c09dfee None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:18.845413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3f40bf8-fa3d-4b68-9eee-6e809c09dfee None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:18.846115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f40bf8-fa3d-4b68-9eee-6e809c09dfee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:18.847467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3f40bf8-fa3d-4b68-9eee-6e809c09dfee None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:18.861659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 520/1027] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Jul 20 19:11:18 2022] GET /identity/v3/services => generated 1957 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:11:19.197820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91220b5-8d80-4b7b-8d38-af717a37a90d None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.198138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91220b5-8d80-4b7b-8d38-af717a37a90d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.198456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91220b5-8d80-4b7b-8d38-af717a37a90d None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.199897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 508/1028] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:19.204928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a1b2-4d72-4091-bd83-489867ff4d7e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.205232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a1b2-4d72-4091-bd83-489867ff4d7e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.205538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0264a1b2-4d72-4091-bd83-489867ff4d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.223958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0264a1b2-4d72-4091-bd83-489867ff4d7e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:19.225199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0264a1b2-4d72-4091-bd83-489867ff4d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:19.295684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 521/1029] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:19.319767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91220b5-8d80-4b7b-8d38-af717a37a90d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.422629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:19.428998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91220b5-8d80-4b7b-8d38-af717a37a90d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.463357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d5f5c-d126-472e-a512-f35bdeb36f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.464104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d5f5c-d126-472e-a512-f35bdeb36f65 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.464388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d5f5c-d126-472e-a512-f35bdeb36f65 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.464665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02d5f5c-d126-472e-a512-f35bdeb36f65 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.475601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c02d5f5c-d126-472e-a512-f35bdeb36f65 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-c02d5f5c-d126-472e-a512-f35bdeb36f65 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:19.484426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:19.485395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:19.486588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 509/1030] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:19.491755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264a1b2-4d72-4091-bd83-489867ff4d7e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.591861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:19.596044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0264a1b2-4d72-4091-bd83-489867ff4d7e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.632431 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca6114-0722-43b2-aef7-f808cc837c43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.633106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca6114-0722-43b2-aef7-f808cc837c43 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.633392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca6114-0722-43b2-aef7-f808cc837c43 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.633701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ca6114-0722-43b2-aef7-f808cc837c43 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.634448 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca6114-0722-43b2-aef7-f808cc837c43 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:19.634848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca6114-0722-43b2-aef7-f808cc837c43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:19.636230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ca6114-0722-43b2-aef7-f808cc837c43 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:19.649059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 522/1031] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:19.650542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d5f5c-d126-472e-a512-f35bdeb36f65 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.753228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02d5f5c-d126-472e-a512-f35bdeb36f65 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.788213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369618f0-10de-4b36-8272-7f61ee700f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.789011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369618f0-10de-4b36-8272-7f61ee700f8f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.789297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369618f0-10de-4b36-8272-7f61ee700f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.789612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-369618f0-10de-4b36-8272-7f61ee700f8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.801287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369618f0-10de-4b36-8272-7f61ee700f8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:19.802770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-369618f0-10de-4b36-8272-7f61ee700f8f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-369618f0-10de-4b36-8272-7f61ee700f8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:19.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:19.821285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:19.822368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:19.822368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:19.822368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:19.822368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:19.825841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 510/1032] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:19.827257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca6114-0722-43b2-aef7-f808cc837c43 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:19.933939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ca6114-0722-43b2-aef7-f808cc837c43 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:19.973750 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c290a-76af-47db-bcbf-a19ebaa3a09d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:19.974432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c290a-76af-47db-bcbf-a19ebaa3a09d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:19.974718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c290a-76af-47db-bcbf-a19ebaa3a09d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:19.974995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-725c290a-76af-47db-bcbf-a19ebaa3a09d None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:19.975737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c290a-76af-47db-bcbf-a19ebaa3a09d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:19.976046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c290a-76af-47db-bcbf-a19ebaa3a09d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:19.977455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-725c290a-76af-47db-bcbf-a19ebaa3a09d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:19.995988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 523/1033] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:19.997265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369618f0-10de-4b36-8272-7f61ee700f8f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.010693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.100019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-369618f0-10de-4b36-8272-7f61ee700f8f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.133524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a4a87b-df17-4c4b-94a8-aaeb40b72f3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.134341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4a87b-df17-4c4b-94a8-aaeb40b72f3b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.135191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4a87b-df17-4c4b-94a8-aaeb40b72f3b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.135517 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a4a87b-df17-4c4b-94a8-aaeb40b72f3b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.143943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4a87b-df17-4c4b-94a8-aaeb40b72f3b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.145374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a4a87b-df17-4c4b-94a8-aaeb40b72f3b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e2a4a87b-df17-4c4b-94a8-aaeb40b72f3b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:20.159671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:20.160623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:20.167029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 511/1034] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Jul 20 19:11:19 2022] GET /identity/v3/users/nova => generated 82 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:20.167832 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c290a-76af-47db-bcbf-a19ebaa3a09d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.171171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192eca-7a0f-4688-aa88-97504c0c8bc5 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.171498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192eca-7a0f-4688-aa88-97504c0c8bc5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.171736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8192eca-7a0f-4688-aa88-97504c0c8bc5 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.173085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 512/1035] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:20.177885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f91595-00c1-41ed-9536-31919fc5f8b2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.178170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f91595-00c1-41ed-9536-31919fc5f8b2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.178444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f91595-00c1-41ed-9536-31919fc5f8b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.193047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.195351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-48f91595-00c1-41ed-9536-31919fc5f8b2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:20.196639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-48f91595-00c1-41ed-9536-31919fc5f8b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:20.259182 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.266083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 513/1036] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:20.276142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-725c290a-76af-47db-bcbf-a19ebaa3a09d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.290336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f91595-00c1-41ed-9536-31919fc5f8b2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.312698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a46d1-ddd7-4fe9-b8f0-3ee1c4897039 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.313376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a46d1-ddd7-4fe9-b8f0-3ee1c4897039 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.313690 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a46d1-ddd7-4fe9-b8f0-3ee1c4897039 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.313969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac2a46d1-ddd7-4fe9-b8f0-3ee1c4897039 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.314806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a46d1-ddd7-4fe9-b8f0-3ee1c4897039 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:20.315120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a46d1-ddd7-4fe9-b8f0-3ee1c4897039 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.316563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac2a46d1-ddd7-4fe9-b8f0-3ee1c4897039 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.343614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 524/1037] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:20.344643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a46d1-ddd7-4fe9-b8f0-3ee1c4897039 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.409419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f91595-00c1-41ed-9536-31919fc5f8b2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.444759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.446653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b515d-df2d-4583-bfb7-5c2a4ab37426 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.448286 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b515d-df2d-4583-bfb7-5c2a4ab37426 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.448709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b515d-df2d-4583-bfb7-5c2a4ab37426 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.448900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-000b515d-df2d-4583-bfb7-5c2a4ab37426 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.450102 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b515d-df2d-4583-bfb7-5c2a4ab37426 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.451540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-000b515d-df2d-4583-bfb7-5c2a4ab37426 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.453330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 514/1038] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:20.458639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b515d-df2d-4583-bfb7-5c2a4ab37426 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.468220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac2a46d1-ddd7-4fe9-b8f0-3ee1c4897039 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.511157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee8fd07-223d-4d6f-b6a7-7130aa58ca73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.511806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8fd07-223d-4d6f-b6a7-7130aa58ca73 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.512087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8fd07-223d-4d6f-b6a7-7130aa58ca73 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.512359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee8fd07-223d-4d6f-b6a7-7130aa58ca73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.532146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.569551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-000b515d-df2d-4583-bfb7-5c2a4ab37426 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.591322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8fd07-223d-4d6f-b6a7-7130aa58ca73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.592758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ee8fd07-223d-4d6f-b6a7-7130aa58ca73 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.606419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006380bf-b80a-4f87-972a-fd909285c144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.607093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006380bf-b80a-4f87-972a-fd909285c144 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.607372 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006380bf-b80a-4f87-972a-fd909285c144 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.607644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006380bf-b80a-4f87-972a-fd909285c144 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.608345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006380bf-b80a-4f87-972a-fd909285c144 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.609798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006380bf-b80a-4f87-972a-fd909285c144 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.631775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 515/1039] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:11:20 2022] POST /identity/v3/users => generated 285 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:11:20.700881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.726807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 525/1040] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:20.733962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006380bf-b80a-4f87-972a-fd909285c144 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.864083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006380bf-b80a-4f87-972a-fd909285c144 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:20.902923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:20.916400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953633de-19d2-434e-9e8d-f2f8a032941f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:20.917306 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953633de-19d2-434e-9e8d-f2f8a032941f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.917651 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953633de-19d2-434e-9e8d-f2f8a032941f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.917938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953633de-19d2-434e-9e8d-f2f8a032941f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.928922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c4beb-07db-49a8-a489-33158b9d3533 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.929460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c4beb-07db-49a8-a489-33158b9d3533 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.929460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90c4beb-07db-49a8-a489-33158b9d3533 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.931036 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 526/1041] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:20.935533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953633de-19d2-434e-9e8d-f2f8a032941f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:20.936625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47cb5-3fa1-4833-8594-f30f8d3d04e2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:20.936961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47cb5-3fa1-4833-8594-f30f8d3d04e2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:20.937063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953633de-19d2-434e-9e8d-f2f8a032941f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:20.937237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47cb5-3fa1-4833-8594-f30f8d3d04e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-953633de-19d2-434e-9e8d-f2f8a032941f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:20.955835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:20.956839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:20.957786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:20.957786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:20.957786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:20.957786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:20.957786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 516/1042] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:20.963358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953633de-19d2-434e-9e8d-f2f8a032941f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:20.963519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-23a47cb5-3fa1-4833-8594-f30f8d3d04e2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:20.964585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-23a47cb5-3fa1-4833-8594-f30f8d3d04e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:21.053171 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 527/1043] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:21.076674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a47cb5-3fa1-4833-8594-f30f8d3d04e2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.086119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953633de-19d2-434e-9e8d-f2f8a032941f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.123893 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78160c21-2985-406f-b298-533bbd6fef5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.124463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78160c21-2985-406f-b298-533bbd6fef5d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.124748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78160c21-2985-406f-b298-533bbd6fef5d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.125027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78160c21-2985-406f-b298-533bbd6fef5d None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.125783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78160c21-2985-406f-b298-533bbd6fef5d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:21.126086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78160c21-2985-406f-b298-533bbd6fef5d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.127455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78160c21-2985-406f-b298-533bbd6fef5d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.150878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 517/1044] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:21.152095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78160c21-2985-406f-b298-533bbd6fef5d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.188973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.196651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a47cb5-3fa1-4833-8594-f30f8d3d04e2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.209682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.235131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5584935-7318-4b76-9a8d-8440cc363762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.235930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5584935-7318-4b76-9a8d-8440cc363762 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.236213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5584935-7318-4b76-9a8d-8440cc363762 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.236514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5584935-7318-4b76-9a8d-8440cc363762 None admin] PATH_INFO: `/v3/services/image` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.237157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5584935-7318-4b76-9a8d-8440cc363762 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.238580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5584935-7318-4b76-9a8d-8440cc363762 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-c5584935-7318-4b76-9a8d-8440cc363762 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:21.251869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:21.252948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 20 19:11:21.253886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:21.258287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 528/1045] 10.222.0.69 () {56 vars in 1105 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/services/image => generated 86 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:21.259059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5584935-7318-4b76-9a8d-8440cc363762 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.269805 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78160c21-2985-406f-b298-533bbd6fef5d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.307884 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17e304-6b5e-45ae-ad2c-389a5c46911a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.308685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17e304-6b5e-45ae-ad2c-389a5c46911a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.308972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17e304-6b5e-45ae-ad2c-389a5c46911a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.309251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d17e304-6b5e-45ae-ad2c-389a5c46911a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.317452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17e304-6b5e-45ae-ad2c-389a5c46911a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.319212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d17e304-6b5e-45ae-ad2c-389a5c46911a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-9d17e304-6b5e-45ae-ad2c-389a5c46911a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:21.335307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:21.336294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:21.337404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:21.337404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:21.337404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:21.337404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:21.337404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:21.337404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:21.337404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:21.337404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:21.337404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 518/1046] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:21.342434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17e304-6b5e-45ae-ad2c-389a5c46911a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.375706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5584935-7318-4b76-9a8d-8440cc363762 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.413624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b23b3-283b-4bfc-9a41-f4df23b7fd22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.414295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b23b3-283b-4bfc-9a41-f4df23b7fd22 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.414921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b23b3-283b-4bfc-9a41-f4df23b7fd22 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.414921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b23b3-283b-4bfc-9a41-f4df23b7fd22 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.415548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b23b3-283b-4bfc-9a41-f4df23b7fd22 None admin] RBAC: Adding query filter params (name=image) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:21.415909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b23b3-283b-4bfc-9a41-f4df23b7fd22 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.417381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b23b3-283b-4bfc-9a41-f4df23b7fd22 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.423320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 529/1047] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/services?name=image => generated 122 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:21.428805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b23b3-283b-4bfc-9a41-f4df23b7fd22 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.450549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.458567 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d17e304-6b5e-45ae-ad2c-389a5c46911a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.471792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.503463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605e823-28cd-47e1-bf28-07a03fd49a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.503463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605e823-28cd-47e1-bf28-07a03fd49a12 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.503463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605e823-28cd-47e1-bf28-07a03fd49a12 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.503463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1605e823-28cd-47e1-bf28-07a03fd49a12 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.503963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605e823-28cd-47e1-bf28-07a03fd49a12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:21.503963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605e823-28cd-47e1-bf28-07a03fd49a12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.505023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1605e823-28cd-47e1-bf28-07a03fd49a12 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.527970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 519/1048] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:21.528225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605e823-28cd-47e1-bf28-07a03fd49a12 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.549785 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b23b3-283b-4bfc-9a41-f4df23b7fd22 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.595584 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d27d6f-1006-4ea0-bf0b-77c123b351b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.596600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d27d6f-1006-4ea0-bf0b-77c123b351b3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.597088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d27d6f-1006-4ea0-bf0b-77c123b351b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.597581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01d27d6f-1006-4ea0-bf0b-77c123b351b3 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.599287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d27d6f-1006-4ea0-bf0b-77c123b351b3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:21.599287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d27d6f-1006-4ea0-bf0b-77c123b351b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.600434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01d27d6f-1006-4ea0-bf0b-77c123b351b3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.611483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 530/1049] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/services?type=image => generated 350 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:21.612301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d27d6f-1006-4ea0-bf0b-77c123b351b3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:21.660029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1605e823-28cd-47e1-bf28-07a03fd49a12 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.705189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de479-164f-438d-af59-d6d2394c5870 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.706720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de479-164f-438d-af59-d6d2394c5870 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.707008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de479-164f-438d-af59-d6d2394c5870 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.707289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de479-164f-438d-af59-d6d2394c5870 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.708133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de479-164f-438d-af59-d6d2394c5870 None admin] RBAC: Adding query filter params (role.id=3e0a48dd676e4ee0b8f3cad1dbe68a6b, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=36955d7d4505457696ab6177adb999a9) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:21.708449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de479-164f-438d-af59-d6d2394c5870 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.709936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de479-164f-438d-af59-d6d2394c5870 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.717894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 520/1050] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:21 2022] GET /identity/v3/role_assignments?user.id=36955d7d4505457696ab6177adb999a9&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=3e0a48dd676e4ee0b8f3cad1dbe68a6b => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:21.754048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01d27d6f-1006-4ea0-bf0b-77c123b351b3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:21.756156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:21.797570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c346bde-e747-4719-9e60-0dc233c6425d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:21.798253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c346bde-e747-4719-9e60-0dc233c6425d None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:21.798540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c346bde-e747-4719-9e60-0dc233c6425d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:21.798820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c346bde-e747-4719-9e60-0dc233c6425d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:21.799586 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c346bde-e747-4719-9e60-0dc233c6425d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:21.800977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c346bde-e747-4719-9e60-0dc233c6425d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:21.829549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 531/1051] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Jul 20 19:11:21 2022] POST /identity/v3/endpoints => generated 327 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:22.811818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebe71d-97c0-41a4-8f63-f157334c5adc None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:22.812365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebe71d-97c0-41a4-8f63-f157334c5adc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:22.812365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ebe71d-97c0-41a4-8f63-f157334c5adc None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:22.813796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 521/1052] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:22.819027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ae40f-d816-4095-9267-50525d64c48f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:22.819418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ae40f-d816-4095-9267-50525d64c48f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:22.819582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-629ae40f-d816-4095-9267-50525d64c48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:22.838107 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-629ae40f-d816-4095-9267-50525d64c48f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:22.839563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-629ae40f-d816-4095-9267-50525d64c48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:22.915250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 532/1053] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:22.939501 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebe71d-97c0-41a4-8f63-f157334c5adc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:22.963005 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:23.053429 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ebe71d-97c0-41a4-8f63-f157334c5adc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:23.090233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad042c-8e4c-4c5f-98e0-7d0b0ae1b701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:23.090951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad042c-8e4c-4c5f-98e0-7d0b0ae1b701 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.091236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad042c-8e4c-4c5f-98e0-7d0b0ae1b701 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.091570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fad042c-8e4c-4c5f-98e0-7d0b0ae1b701 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.102236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fad042c-8e4c-4c5f-98e0-7d0b0ae1b701 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-8fad042c-8e4c-4c5f-98e0-7d0b0ae1b701 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:23.110180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:23.111178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:23.112074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:23.114016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 522/1054] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:23.117532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ae40f-d816-4095-9267-50525d64c48f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.204815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:23.221915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-629ae40f-d816-4095-9267-50525d64c48f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:23.258354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c012a-5728-4c38-b8b5-23b1f92866f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:23.259027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c012a-5728-4c38-b8b5-23b1f92866f6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.259314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c012a-5728-4c38-b8b5-23b1f92866f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.259665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e17c012a-5728-4c38-b8b5-23b1f92866f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.260416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c012a-5728-4c38-b8b5-23b1f92866f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:23.260729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c012a-5728-4c38-b8b5-23b1f92866f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:23.262161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e17c012a-5728-4c38-b8b5-23b1f92866f6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:23.272256 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 533/1055] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:23.277176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad042c-8e4c-4c5f-98e0-7d0b0ae1b701 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.373157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:23.394704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fad042c-8e4c-4c5f-98e0-7d0b0ae1b701 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:23.430189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e444bdc-f139-4d36-8a42-112aba30c275 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:23.431069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e444bdc-f139-4d36-8a42-112aba30c275 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.431896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e444bdc-f139-4d36-8a42-112aba30c275 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.432171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e444bdc-f139-4d36-8a42-112aba30c275 None admin] PATH_INFO: `/v3/users/63a7a0e97ec441198ff7c4f9f6604c83` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.446126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e444bdc-f139-4d36-8a42-112aba30c275 None admin] RBAC: Authorizing `identity:get_user(user_id=63a7a0e97ec441198ff7c4f9f6604c83)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:23.448371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e444bdc-f139-4d36-8a42-112aba30c275 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:23.450338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 523/1056] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:23 2022] GET /identity/v3/users/63a7a0e97ec441198ff7c4f9f6604c83 => generated 285 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:23.456085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c012a-5728-4c38-b8b5-23b1f92866f6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.561025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e17c012a-5728-4c38-b8b5-23b1f92866f6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:23.595025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fe804-a154-44b8-a2e4-e57a337b6ded None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:23.595678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fe804-a154-44b8-a2e4-e57a337b6ded None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.595960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fe804-a154-44b8-a2e4-e57a337b6ded None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.596235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8fe804-a154-44b8-a2e4-e57a337b6ded None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.613104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:23.668187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fe804-a154-44b8-a2e4-e57a337b6ded None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:23.669753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8fe804-a154-44b8-a2e4-e57a337b6ded None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:23.770836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e5df0-8bfe-4a93-bf35-a411d556f55f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.771123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e5df0-8bfe-4a93-bf35-a411d556f55f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.771401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b43e5df0-8bfe-4a93-bf35-a411d556f55f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.772778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 524/1057] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:23.778489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734996-2c75-4da8-9340-5f7a9a5de163 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.778785 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734996-2c75-4da8-9340-5f7a9a5de163 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.779065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7734996-2c75-4da8-9340-5f7a9a5de163 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.786685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 534/1058] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:23.787756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fe804-a154-44b8-a2e4-e57a337b6ded None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.799596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c7734996-2c75-4da8-9340-5f7a9a5de163 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:23.800781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c7734996-2c75-4da8-9340-5f7a9a5de163 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:23.835929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:23.872069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 525/1059] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:23.895032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7734996-2c75-4da8-9340-5f7a9a5de163 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.895529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8fe804-a154-44b8-a2e4-e57a337b6ded None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:23.933295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45dab65e-1c9d-4141-8c4d-9f783094bcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:23.934115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dab65e-1c9d-4141-8c4d-9f783094bcc7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.934400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dab65e-1c9d-4141-8c4d-9f783094bcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.934674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45dab65e-1c9d-4141-8c4d-9f783094bcc7 None admin] PATH_INFO: `/v3/projects/075d92d1feca4858ab197d1421621399` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.935903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dab65e-1c9d-4141-8c4d-9f783094bcc7 None admin] RBAC: Authorizing `identity:get_project(project_id=075d92d1feca4858ab197d1421621399)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:23.937347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45dab65e-1c9d-4141-8c4d-9f783094bcc7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:23.939135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 535/1060] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:23 2022] GET /identity/v3/projects/075d92d1feca4858ab197d1421621399 => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:23.942873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17f19a-7e10-46cf-b123-260ba41d2876 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:23.943231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17f19a-7e10-46cf-b123-260ba41d2876 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:23.943524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17f19a-7e10-46cf-b123-260ba41d2876 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:23.944798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 536/1061] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:23.945491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17f19a-7e10-46cf-b123-260ba41d2876 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:23.963620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.000883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7734996-2c75-4da8-9340-5f7a9a5de163 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.034356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef8b337-d884-4963-9010-f94dc0b6c73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.035159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8b337-d884-4963-9010-f94dc0b6c73a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.035438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8b337-d884-4963-9010-f94dc0b6c73a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.035709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef8b337-d884-4963-9010-f94dc0b6c73a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.046391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef8b337-d884-4963-9010-f94dc0b6c73a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:24.049371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17f19a-7e10-46cf-b123-260ba41d2876 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-cef8b337-d884-4963-9010-f94dc0b6c73a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:24.054270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:24.055305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:24.056329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 526/1062] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:24.057828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5d94c-25b0-427d-bd03-3291a3c786a5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.058118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5d94c-25b0-427d-bd03-3291a3c786a5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.058688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa5d94c-25b0-427d-bd03-3291a3c786a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.065741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.075662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-faa5d94c-25b0-427d-bd03-3291a3c786a5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:24.076848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-faa5d94c-25b0-427d-bd03-3291a3c786a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:24.086456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b427b-2ece-401a-a901-2c42fe5bfaca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.087109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b427b-2ece-401a-a901-2c42fe5bfaca None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.087442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b427b-2ece-401a-a901-2c42fe5bfaca None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.087838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385b427b-2ece-401a-a901-2c42fe5bfaca None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.088661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b427b-2ece-401a-a901-2c42fe5bfaca None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=075d92d1feca4858ab197d1421621399, user.id=63a7a0e97ec441198ff7c4f9f6604c83) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:24.088977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b427b-2ece-401a-a901-2c42fe5bfaca None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.090604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385b427b-2ece-401a-a901-2c42fe5bfaca None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.096383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 537/1063] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:23 2022] GET /identity/v3/role_assignments?user.id=63a7a0e97ec441198ff7c4f9f6604c83&scope.project.id=075d92d1feca4858ab197d1421621399&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:24.097161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b427b-2ece-401a-a901-2c42fe5bfaca None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.164165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 527/1064] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:24.188899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5d94c-25b0-427d-bd03-3291a3c786a5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.214568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385b427b-2ece-401a-a901-2c42fe5bfaca None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.253892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.255936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdf308-2146-425b-8ad1-9584b6bbd72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.256649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdf308-2146-425b-8ad1-9584b6bbd72d None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.256953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdf308-2146-425b-8ad1-9584b6bbd72d None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.257223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbdf308-2146-425b-8ad1-9584b6bbd72d None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.258011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdf308-2146-425b-8ad1-9584b6bbd72d None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:24.258322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdf308-2146-425b-8ad1-9584b6bbd72d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.260565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbdf308-2146-425b-8ad1-9584b6bbd72d None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.275838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 538/1065] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:24.277196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdf308-2146-425b-8ad1-9584b6bbd72d None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.299780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa5d94c-25b0-427d-bd03-3291a3c786a5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.322988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.345249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d4dc0-7491-4b4d-901e-49d3b2705a16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.346029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d4dc0-7491-4b4d-901e-49d3b2705a16 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.346555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d4dc0-7491-4b4d-901e-49d3b2705a16 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.346555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-526d4dc0-7491-4b4d-901e-49d3b2705a16 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.389017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbdf308-2146-425b-8ad1-9584b6bbd72d None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.420772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d4dc0-7491-4b4d-901e-49d3b2705a16 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.422433 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-526d4dc0-7491-4b4d-901e-49d3b2705a16 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.425281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c3ed8-e193-4b27-b159-8bd994d92746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.426098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c3ed8-e193-4b27-b159-8bd994d92746 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.426423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c3ed8-e193-4b27-b159-8bd994d92746 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.426704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-232c3ed8-e193-4b27-b159-8bd994d92746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.438844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c3ed8-e193-4b27-b159-8bd994d92746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.440361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-232c3ed8-e193-4b27-b159-8bd994d92746 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-232c3ed8-e193-4b27-b159-8bd994d92746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:24.462458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:24.463379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:24.464283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:24.464283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:24.464283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:24.464283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:24.464283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 539/1066] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:24.469976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c3ed8-e193-4b27-b159-8bd994d92746 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.486736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.538540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 528/1067] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:11:24.539342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d4dc0-7491-4b4d-901e-49d3b2705a16 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.572881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.583407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-232c3ed8-e193-4b27-b159-8bd994d92746 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.620947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673c3d1-1f84-4ae1-b811-74314d0fd7dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.621638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673c3d1-1f84-4ae1-b811-74314d0fd7dc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.621928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673c3d1-1f84-4ae1-b811-74314d0fd7dc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.622206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7673c3d1-1f84-4ae1-b811-74314d0fd7dc None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.622927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673c3d1-1f84-4ae1-b811-74314d0fd7dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:24.623235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673c3d1-1f84-4ae1-b811-74314d0fd7dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.624601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7673c3d1-1f84-4ae1-b811-74314d0fd7dc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.636993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 540/1068] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:24.642766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673c3d1-1f84-4ae1-b811-74314d0fd7dc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.649021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-526d4dc0-7491-4b4d-901e-49d3b2705a16 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.687533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f9f93-1a7f-4f7e-be64-15b7a23a14db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.688662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f9f93-1a7f-4f7e-be64-15b7a23a14db None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.689148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f9f93-1a7f-4f7e-be64-15b7a23a14db None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.689657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-310f9f93-1a7f-4f7e-be64-15b7a23a14db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.701803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f9f93-1a7f-4f7e-be64-15b7a23a14db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.703550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-310f9f93-1a7f-4f7e-be64-15b7a23a14db None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-310f9f93-1a7f-4f7e-be64-15b7a23a14db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:24.722224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:24.723448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:24.724489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:24.724489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:24.724489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:24.724489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:24.726077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 529/1069] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:24.732336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f9f93-1a7f-4f7e-be64-15b7a23a14db None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.746744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.754300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7673c3d1-1f84-4ae1-b811-74314d0fd7dc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.791289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f5943-5582-4631-8592-3d6a19c12ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.792380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f5943-5582-4631-8592-3d6a19c12ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.792972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f5943-5582-4631-8592-3d6a19c12ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.793471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65f5943-5582-4631-8592-3d6a19c12ad2 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.801077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f5943-5582-4631-8592-3d6a19c12ad2 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.802760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65f5943-5582-4631-8592-3d6a19c12ad2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-d65f5943-5582-4631-8592-3d6a19c12ad2 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:24.820960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:24.822432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:24.823782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:24.823782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:24.823782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:24.823782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:24.823782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:24.823782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:24.823782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:24.823782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:24.823782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:24.823782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:24.825650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 541/1070] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:24.828230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f5943-5582-4631-8592-3d6a19c12ad2 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:24.844080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:24.859110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-310f9f93-1a7f-4f7e-be64-15b7a23a14db None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:24.895702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502da742-4e9f-41a8-a9d5-5606b8df891e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:24.896465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502da742-4e9f-41a8-a9d5-5606b8df891e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:24.896820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502da742-4e9f-41a8-a9d5-5606b8df891e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:24.897105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502da742-4e9f-41a8-a9d5-5606b8df891e None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:24.897870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502da742-4e9f-41a8-a9d5-5606b8df891e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:24.898179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502da742-4e9f-41a8-a9d5-5606b8df891e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:24.899574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502da742-4e9f-41a8-a9d5-5606b8df891e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:24.921549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 530/1071] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:24.960039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65f5943-5582-4631-8592-3d6a19c12ad2 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.000314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66b9ca-8993-4798-9a93-c866c2add8e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.001001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66b9ca-8993-4798-9a93-c866c2add8e9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.001352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66b9ca-8993-4798-9a93-c866c2add8e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.001628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b66b9ca-8993-4798-9a93-c866c2add8e9 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.002539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66b9ca-8993-4798-9a93-c866c2add8e9 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:25.002850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66b9ca-8993-4798-9a93-c866c2add8e9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.004303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b66b9ca-8993-4798-9a93-c866c2add8e9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.025417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 542/1072] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Jul 20 19:11:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:25.031494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502da742-4e9f-41a8-a9d5-5606b8df891e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.112621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:25.157244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502da742-4e9f-41a8-a9d5-5606b8df891e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.194851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2a14c-c6cc-44b4-b9b4-5f7bea3a418e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.195671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2a14c-c6cc-44b4-b9b4-5f7bea3a418e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.195975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2a14c-c6cc-44b4-b9b4-5f7bea3a418e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.196254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d2a14c-c6cc-44b4-b9b4-5f7bea3a418e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.209221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2a14c-c6cc-44b4-b9b4-5f7bea3a418e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.210827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d2a14c-c6cc-44b4-b9b4-5f7bea3a418e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a5d2a14c-c6cc-44b4-b9b4-5f7bea3a418e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:25.230913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:25.232096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:25.233234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:25.233234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:25.233234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:25.233234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:25.233234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 531/1073] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:25.238143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66b9ca-8993-4798-9a93-c866c2add8e9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.307992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:25.342147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b66b9ca-8993-4798-9a93-c866c2add8e9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.376305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335b64f-1437-4913-8a5b-a35ab6da7ac8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.376958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335b64f-1437-4913-8a5b-a35ab6da7ac8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.377243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335b64f-1437-4913-8a5b-a35ab6da7ac8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.377537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2335b64f-1437-4913-8a5b-a35ab6da7ac8 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.378267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335b64f-1437-4913-8a5b-a35ab6da7ac8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:25.378576 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335b64f-1437-4913-8a5b-a35ab6da7ac8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.379935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2335b64f-1437-4913-8a5b-a35ab6da7ac8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.390986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 543/1074] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:25.396507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2a14c-c6cc-44b4-b9b4-5f7bea3a418e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.498028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d2a14c-c6cc-44b4-b9b4-5f7bea3a418e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.520174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:25.532224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354e57a-eb32-44c1-a610-47d05c3ca1f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.533020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e57a-eb32-44c1-a610-47d05c3ca1f2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.533301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e57a-eb32-44c1-a610-47d05c3ca1f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.533603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e57a-eb32-44c1-a610-47d05c3ca1f2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.541051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354e57a-eb32-44c1-a610-47d05c3ca1f2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.542982 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354e57a-eb32-44c1-a610-47d05c3ca1f2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-e354e57a-eb32-44c1-a610-47d05c3ca1f2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:25.557774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:25.558939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:25.559870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:25.559870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:25.559870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:25.559870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:25.559870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:25.559870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:25.559870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:25.559870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:25.559870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 532/1075] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:25.564760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335b64f-1437-4913-8a5b-a35ab6da7ac8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.673704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2335b64f-1437-4913-8a5b-a35ab6da7ac8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.709682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:25.717133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dfcfe85-b5ca-46c7-b55c-d9e1af007237 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.717841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcfe85-b5ca-46c7-b55c-d9e1af007237 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.718103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcfe85-b5ca-46c7-b55c-d9e1af007237 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.718456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dfcfe85-b5ca-46c7-b55c-d9e1af007237 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.719241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcfe85-b5ca-46c7-b55c-d9e1af007237 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:25.719544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcfe85-b5ca-46c7-b55c-d9e1af007237 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.721004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dfcfe85-b5ca-46c7-b55c-d9e1af007237 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.736418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 544/1076] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:25.742840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354e57a-eb32-44c1-a610-47d05c3ca1f2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:25.858035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354e57a-eb32-44c1-a610-47d05c3ca1f2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:25.896617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e017da-7175-4c7e-ac87-dc0664a9a0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:25.897531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e017da-7175-4c7e-ac87-dc0664a9a0aa None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:25.897824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e017da-7175-4c7e-ac87-dc0664a9a0aa None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:25.898100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e017da-7175-4c7e-ac87-dc0664a9a0aa None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/36955d7d4505457696ab6177adb999a9/roles/3e0a48dd676e4ee0b8f3cad1dbe68a6b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:25.900944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e017da-7175-4c7e-ac87-dc0664a9a0aa None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=36955d7d4505457696ab6177adb999a9, role_id=3e0a48dd676e4ee0b8f3cad1dbe68a6b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:25.902521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e017da-7175-4c7e-ac87-dc0664a9a0aa None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:25.916133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 533/1077] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:25 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/36955d7d4505457696ab6177adb999a9/roles/3e0a48dd676e4ee0b8f3cad1dbe68a6b => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:11:26.155714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8f81a-1ee2-4eca-8352-d0b94e725e66 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.156310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8f81a-1ee2-4eca-8352-d0b94e725e66 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.156310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b8f81a-1ee2-4eca-8352-d0b94e725e66 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.157839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 545/1078] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:26.163290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efe9ec-d280-4c18-bba8-faf7948db977 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.163597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efe9ec-d280-4c18-bba8-faf7948db977 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.163870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29efe9ec-d280-4c18-bba8-faf7948db977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.182284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-29efe9ec-d280-4c18-bba8-faf7948db977 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:26.183493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-29efe9ec-d280-4c18-bba8-faf7948db977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:26.220692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:26.259188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 534/1079] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:26.282714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8f81a-1ee2-4eca-8352-d0b94e725e66 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.389672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b8f81a-1ee2-4eca-8352-d0b94e725e66 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.423461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79539b0f-c676-4893-92fd-dce5fa946ec5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.424165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79539b0f-c676-4893-92fd-dce5fa946ec5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.424442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79539b0f-c676-4893-92fd-dce5fa946ec5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.424712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79539b0f-c676-4893-92fd-dce5fa946ec5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.435208 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79539b0f-c676-4893-92fd-dce5fa946ec5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-79539b0f-c676-4893-92fd-dce5fa946ec5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:26.442658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:26.443767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:26.444622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 546/1080] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:26.450562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efe9ec-d280-4c18-bba8-faf7948db977 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.548432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29efe9ec-d280-4c18-bba8-faf7948db977 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.582112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0403a-5df3-4fb0-adc6-a81c665ffa42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.582774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0403a-5df3-4fb0-adc6-a81c665ffa42 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.583052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0403a-5df3-4fb0-adc6-a81c665ffa42 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.583322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc0403a-5df3-4fb0-adc6-a81c665ffa42 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.584057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0403a-5df3-4fb0-adc6-a81c665ffa42 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:26.584361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0403a-5df3-4fb0-adc6-a81c665ffa42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:26.585746 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cc0403a-5df3-4fb0-adc6-a81c665ffa42 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:26.594725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 535/1081] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:26.601088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79539b0f-c676-4893-92fd-dce5fa946ec5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.651508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:26.701020 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79539b0f-c676-4893-92fd-dce5fa946ec5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.734098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea91a03-1708-4aa1-88ea-c1a2ec556195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.734947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea91a03-1708-4aa1-88ea-c1a2ec556195 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.735223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea91a03-1708-4aa1-88ea-c1a2ec556195 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.735491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea91a03-1708-4aa1-88ea-c1a2ec556195 None admin] PATH_INFO: `/v3/users/63a7a0e97ec441198ff7c4f9f6604c83` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.736653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea91a03-1708-4aa1-88ea-c1a2ec556195 None admin] RBAC: Authorizing `identity:get_user(user_id=63a7a0e97ec441198ff7c4f9f6604c83)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:26.738243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ea91a03-1708-4aa1-88ea-c1a2ec556195 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:26.739720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 547/1082] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:26 2022] GET /identity/v3/users/63a7a0e97ec441198ff7c4f9f6604c83 => generated 285 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:26.746368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0403a-5df3-4fb0-adc6-a81c665ffa42 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.780855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:26.850376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc0403a-5df3-4fb0-adc6-a81c665ffa42 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:26.882698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3199a09-1ccb-4687-923e-29dd0f913d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:26.883551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3199a09-1ccb-4687-923e-29dd0f913d31 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:26.883833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3199a09-1ccb-4687-923e-29dd0f913d31 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:26.884108 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3199a09-1ccb-4687-923e-29dd0f913d31 None admin] PATH_INFO: `/v3/projects/075d92d1feca4858ab197d1421621399` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:26.885929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3199a09-1ccb-4687-923e-29dd0f913d31 None admin] RBAC: Authorizing `identity:get_project(project_id=075d92d1feca4858ab197d1421621399)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:26.887413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3199a09-1ccb-4687-923e-29dd0f913d31 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:26.889133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 536/1083] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:26 2022] GET /identity/v3/projects/075d92d1feca4858ab197d1421621399 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:26.895070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea91a03-1708-4aa1-88ea-c1a2ec556195 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:26.998808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ea91a03-1708-4aa1-88ea-c1a2ec556195 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:27.016251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:27.038247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f778325-d365-48d0-a5f9-5a043cff59a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:27.039147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f778325-d365-48d0-a5f9-5a043cff59a4 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:27.039426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f778325-d365-48d0-a5f9-5a043cff59a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:27.039697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f778325-d365-48d0-a5f9-5a043cff59a4 None admin] PATH_INFO: `/v3/projects/075d92d1feca4858ab197d1421621399/users/63a7a0e97ec441198ff7c4f9f6604c83/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:27.042318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f778325-d365-48d0-a5f9-5a043cff59a4 None admin] RBAC: Authorizing `identity:create_grant(project_id=075d92d1feca4858ab197d1421621399, user_id=63a7a0e97ec441198ff7c4f9f6604c83, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:27.043742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f778325-d365-48d0-a5f9-5a043cff59a4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:27.057291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 548/1084] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:26 2022] PUT /identity/v3/projects/075d92d1feca4858ab197d1421621399/users/63a7a0e97ec441198ff7c4f9f6604c83/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:11:27.968602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f406dc-77c7-4bdc-8277-a12bbdd99440 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:27.969166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f406dc-77c7-4bdc-8277-a12bbdd99440 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:27.969166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f406dc-77c7-4bdc-8277-a12bbdd99440 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:27.970898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 537/1085] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:27.976533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ece89-f44c-43e9-b403-53d631679387 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:27.976945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ece89-f44c-43e9-b403-53d631679387 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:27.977104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-046ece89-f44c-43e9-b403-53d631679387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:27.994680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-046ece89-f44c-43e9-b403-53d631679387 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:27.995872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-046ece89-f44c-43e9-b403-53d631679387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:28.061654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 549/1086] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:28.085005 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f406dc-77c7-4bdc-8277-a12bbdd99440 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:28.188858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f406dc-77c7-4bdc-8277-a12bbdd99440 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:28.190391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:28.224074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e939512-90bd-4b03-87d5-be465cb9e831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:28.224807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e939512-90bd-4b03-87d5-be465cb9e831 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.225094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e939512-90bd-4b03-87d5-be465cb9e831 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.225371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e939512-90bd-4b03-87d5-be465cb9e831 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.236362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e939512-90bd-4b03-87d5-be465cb9e831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-0e939512-90bd-4b03-87d5-be465cb9e831 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:28.244502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:28.245453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:28.246491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 538/1087] 10.222.0.69 () {56 vars in 1120 bytes} [Wed Jul 20 19:11:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:28.251790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ece89-f44c-43e9-b403-53d631679387 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:28.352546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-046ece89-f44c-43e9-b403-53d631679387 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:28.362286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:28.388620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f44215-997d-48bd-8eb1-11a29540f710 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:28.389303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f44215-997d-48bd-8eb1-11a29540f710 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.389604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f44215-997d-48bd-8eb1-11a29540f710 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.389885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f44215-997d-48bd-8eb1-11a29540f710 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.390636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f44215-997d-48bd-8eb1-11a29540f710 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:28.390948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f44215-997d-48bd-8eb1-11a29540f710 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:28.392309 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82f44215-997d-48bd-8eb1-11a29540f710 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:28.400108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 550/1088] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:28.405833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e939512-90bd-4b03-87d5-be465cb9e831 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:28.504877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e939512-90bd-4b03-87d5-be465cb9e831 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:28.538048 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a94c91-1dee-4268-b46b-7fb8431ec70c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:28.538835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a94c91-1dee-4268-b46b-7fb8431ec70c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.539117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a94c91-1dee-4268-b46b-7fb8431ec70c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.539392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a94c91-1dee-4268-b46b-7fb8431ec70c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.549900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a94c91-1dee-4268-b46b-7fb8431ec70c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:28.551339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a94c91-1dee-4268-b46b-7fb8431ec70c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-14a94c91-1dee-4268-b46b-7fb8431ec70c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:28.569049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:28.570046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:28.570957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:28.570957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:28.570957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:28.570957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:28.570957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 539/1089] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:28.576086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f44215-997d-48bd-8eb1-11a29540f710 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:28.677451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f44215-997d-48bd-8eb1-11a29540f710 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:28.711508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67158eb8-950c-45a3-972e-2eb722d613c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:28.712169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67158eb8-950c-45a3-972e-2eb722d613c6 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.712966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67158eb8-950c-45a3-972e-2eb722d613c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.713251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67158eb8-950c-45a3-972e-2eb722d613c6 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.713999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67158eb8-950c-45a3-972e-2eb722d613c6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:28.714308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67158eb8-950c-45a3-972e-2eb722d613c6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:28.715677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67158eb8-950c-45a3-972e-2eb722d613c6 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:28.728719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 551/1090] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:28.734507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a94c91-1dee-4268-b46b-7fb8431ec70c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:28.757339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:28.848865 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a94c91-1dee-4268-b46b-7fb8431ec70c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:28.884473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499f1eff-2c7f-4b8a-a3b8-ba4ce22fdce5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:28.885375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f1eff-2c7f-4b8a-a3b8-ba4ce22fdce5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:28.885689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f1eff-2c7f-4b8a-a3b8-ba4ce22fdce5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:28.885968 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499f1eff-2c7f-4b8a-a3b8-ba4ce22fdce5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:28.893537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f1eff-2c7f-4b8a-a3b8-ba4ce22fdce5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:28.895040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-499f1eff-2c7f-4b8a-a3b8-ba4ce22fdce5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-499f1eff-2c7f-4b8a-a3b8-ba4ce22fdce5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:11:28.914539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:28.916518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:28.917553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:28.917553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:28.917553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:11:28.917553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:11:28.917553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:11:28.917553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:11:28.917553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:11:28.917553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:11:28.917553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 20 19:11:28.917553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:28.921399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 540/1091] 10.222.0.69 () {56 vars in 1093 bytes} [Wed Jul 20 19:11:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:28.922630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67158eb8-950c-45a3-972e-2eb722d613c6 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:28.954150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.026250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67158eb8-950c-45a3-972e-2eb722d613c6 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.059324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e128c85-a1e9-4327-b2f8-1343c45d14a5 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.059839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e128c85-a1e9-4327-b2f8-1343c45d14a5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.059958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e128c85-a1e9-4327-b2f8-1343c45d14a5 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.061389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 541/1092] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:29.061922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938a16f-44e9-4c76-8013-89d30ac03bfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.062594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938a16f-44e9-4c76-8013-89d30ac03bfb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.062872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938a16f-44e9-4c76-8013-89d30ac03bfb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.063142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e938a16f-44e9-4c76-8013-89d30ac03bfb None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.063976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938a16f-44e9-4c76-8013-89d30ac03bfb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:29.064281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938a16f-44e9-4c76-8013-89d30ac03bfb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.065719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e938a16f-44e9-4c76-8013-89d30ac03bfb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.066235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bff38-82ff-4835-a915-bae9a2cada6b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.066523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bff38-82ff-4835-a915-bae9a2cada6b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.066804 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706bff38-82ff-4835-a915-bae9a2cada6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.083834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-706bff38-82ff-4835-a915-bae9a2cada6b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:29.085045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-706bff38-82ff-4835-a915-bae9a2cada6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:29.087143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 552/1093] 10.222.0.69 () {56 vars in 1133 bytes} [Wed Jul 20 19:11:28 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:29.093380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938a16f-44e9-4c76-8013-89d30ac03bfb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.152588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 542/1094] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:29.175901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bff38-82ff-4835-a915-bae9a2cada6b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.178858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.188606 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.201577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e938a16f-44e9-4c76-8013-89d30ac03bfb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.242870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab05b2a-e8cb-4e3c-8921-2821a7813907 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.244358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab05b2a-e8cb-4e3c-8921-2821a7813907 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.244630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab05b2a-e8cb-4e3c-8921-2821a7813907 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.245089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cab05b2a-e8cb-4e3c-8921-2821a7813907 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.278329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706bff38-82ff-4835-a915-bae9a2cada6b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.315222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f5644-a3bc-4ddc-94cf-550606446dea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.318468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f5644-a3bc-4ddc-94cf-550606446dea None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.318468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f5644-a3bc-4ddc-94cf-550606446dea None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.318468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-917f5644-a3bc-4ddc-94cf-550606446dea None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.328938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-917f5644-a3bc-4ddc-94cf-550606446dea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:29.334516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab05b2a-e8cb-4e3c-8921-2821a7813907 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.335980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cab05b2a-e8cb-4e3c-8921-2821a7813907 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-917f5644-a3bc-4ddc-94cf-550606446dea None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:29.336603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:29.338349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:29.339617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 543/1095] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:29.344940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f5644-a3bc-4ddc-94cf-550606446dea None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.410273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.435777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.449735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 553/1096] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:11:29.455547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab05b2a-e8cb-4e3c-8921-2821a7813907 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.458757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-917f5644-a3bc-4ddc-94cf-550606446dea None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.499293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34bc52-cb7d-4cce-b4ef-ea0806842b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.499969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34bc52-cb7d-4cce-b4ef-ea0806842b5b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.500254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34bc52-cb7d-4cce-b4ef-ea0806842b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.500531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a34bc52-cb7d-4cce-b4ef-ea0806842b5b None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.501279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34bc52-cb7d-4cce-b4ef-ea0806842b5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:29.501618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34bc52-cb7d-4cce-b4ef-ea0806842b5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.503173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a34bc52-cb7d-4cce-b4ef-ea0806842b5b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.512456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 544/1097] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:29.518249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34bc52-cb7d-4cce-b4ef-ea0806842b5b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.564543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cab05b2a-e8cb-4e3c-8921-2821a7813907 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.599269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a265238-0719-4540-a13d-fe32ac5642fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.600042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a265238-0719-4540-a13d-fe32ac5642fe None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.600320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a265238-0719-4540-a13d-fe32ac5642fe None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.600585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a265238-0719-4540-a13d-fe32ac5642fe None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.611603 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a265238-0719-4540-a13d-fe32ac5642fe None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.613057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a265238-0719-4540-a13d-fe32ac5642fe None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.620145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a34bc52-cb7d-4cce-b4ef-ea0806842b5b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-1a265238-0719-4540-a13d-fe32ac5642fe None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:29.630479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:29.631442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:29.632338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:11:29.632338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:29.632338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:11:29.632338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:29.632338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 554/1098] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:29.637958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a265238-0719-4540-a13d-fe32ac5642fe None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.650759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.659456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b3b48-21bb-4d76-830f-e62d3b1e13a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.660330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b3b48-21bb-4d76-830f-e62d3b1e13a5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.660609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b3b48-21bb-4d76-830f-e62d3b1e13a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.660883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b3b48-21bb-4d76-830f-e62d3b1e13a5 None admin] PATH_INFO: `/v3/users/63a7a0e97ec441198ff7c4f9f6604c83` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.662112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b3b48-21bb-4d76-830f-e62d3b1e13a5 None admin] RBAC: Authorizing `identity:get_user(user_id=63a7a0e97ec441198ff7c4f9f6604c83)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.663555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b3b48-21bb-4d76-830f-e62d3b1e13a5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.665198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 545/1099] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/users/63a7a0e97ec441198ff7c4f9f6604c83 => generated 285 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:29.671605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b3b48-21bb-4d76-830f-e62d3b1e13a5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.680289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.739332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a265238-0719-4540-a13d-fe32ac5642fe None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.769373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b3b48-21bb-4d76-830f-e62d3b1e13a5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.772486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753cffa-2fa6-4604-8c89-7540a960697b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.773138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753cffa-2fa6-4604-8c89-7540a960697b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.773417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753cffa-2fa6-4604-8c89-7540a960697b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.773725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b753cffa-2fa6-4604-8c89-7540a960697b None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.774442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753cffa-2fa6-4604-8c89-7540a960697b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:29.776263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753cffa-2fa6-4604-8c89-7540a960697b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.777094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b753cffa-2fa6-4604-8c89-7540a960697b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.788239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 555/1100] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:29.794697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753cffa-2fa6-4604-8c89-7540a960697b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.803999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dde5d7-aa3c-4843-a079-1870835a0303 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.804651 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde5d7-aa3c-4843-a079-1870835a0303 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.804934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde5d7-aa3c-4843-a079-1870835a0303 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.805211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dde5d7-aa3c-4843-a079-1870835a0303 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.865975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.874628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dde5d7-aa3c-4843-a079-1870835a0303 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.876085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dde5d7-aa3c-4843-a079-1870835a0303 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.892967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:29.896732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b753cffa-2fa6-4604-8c89-7540a960697b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:29.931550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e984a6-78bb-4967-8c5d-d1ca60c55a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:29.932380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e984a6-78bb-4967-8c5d-d1ca60c55a0e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:29.932661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e984a6-78bb-4967-8c5d-d1ca60c55a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:29.932945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e984a6-78bb-4967-8c5d-d1ca60c55a0e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:29.939803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e984a6-78bb-4967-8c5d-d1ca60c55a0e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:29.941229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e984a6-78bb-4967-8c5d-d1ca60c55a0e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b8e984a6-78bb-4967-8c5d-d1ca60c55a0e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:11:29.955924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:29.956861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:29.958041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:11:29.958041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:11:29.958041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:29.958041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:29.958041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:29.958041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:29.958041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:11:29.958041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:29.958041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 556/1101] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:11:29.963481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e984a6-78bb-4967-8c5d-d1ca60c55a0e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:29.979514 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 546/1102] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:11:29.985118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dde5d7-aa3c-4843-a079-1870835a0303 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.065097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e984a6-78bb-4967-8c5d-d1ca60c55a0e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.076548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:30.084007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dde5d7-aa3c-4843-a079-1870835a0303 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.099693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05683d3-d511-4638-8058-5f61e5caec18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.100353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05683d3-d511-4638-8058-5f61e5caec18 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.100639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05683d3-d511-4638-8058-5f61e5caec18 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.100919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d05683d3-d511-4638-8058-5f61e5caec18 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.101804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05683d3-d511-4638-8058-5f61e5caec18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:30.102121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05683d3-d511-4638-8058-5f61e5caec18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.104103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d05683d3-d511-4638-8058-5f61e5caec18 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.115845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 557/1103] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:30.117875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b637de-e403-4c13-abc5-651ec40f4cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.119427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b637de-e403-4c13-abc5-651ec40f4cf5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.119718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b637de-e403-4c13-abc5-651ec40f4cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.119995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5b637de-e403-4c13-abc5-651ec40f4cf5 None admin] PATH_INFO: `/v3/projects/075d92d1feca4858ab197d1421621399` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.121283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b637de-e403-4c13-abc5-651ec40f4cf5 None admin] RBAC: Authorizing `identity:get_project(project_id=075d92d1feca4858ab197d1421621399)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.122291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05683d3-d511-4638-8058-5f61e5caec18 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.122823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5b637de-e403-4c13-abc5-651ec40f4cf5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.125539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 547/1104] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:29 2022] GET /identity/v3/projects/075d92d1feca4858ab197d1421621399 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:30.130318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b637de-e403-4c13-abc5-651ec40f4cf5 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:30.145431 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:30.226141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d05683d3-d511-4638-8058-5f61e5caec18 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.237725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5b637de-e403-4c13-abc5-651ec40f4cf5 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:30.259178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6ae2a2-3984-465c-b162-ec3dc59609c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.259947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ae2a2-3984-465c-b162-ec3dc59609c4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.260225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ae2a2-3984-465c-b162-ec3dc59609c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.260497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6ae2a2-3984-465c-b162-ec3dc59609c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.261319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ae2a2-3984-465c-b162-ec3dc59609c4 None admin] RBAC: Adding query filter params (role.id=3e0a48dd676e4ee0b8f3cad1dbe68a6b, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=36955d7d4505457696ab6177adb999a9) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:30.261668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ae2a2-3984-465c-b162-ec3dc59609c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.263078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6ae2a2-3984-465c-b162-ec3dc59609c4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.268720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 558/1105] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/role_assignments?user.id=36955d7d4505457696ab6177adb999a9&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=3e0a48dd676e4ee0b8f3cad1dbe68a6b => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:30.270876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f58eead8-70d8-436d-af1e-e1077389bac3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:30.271529 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eead8-70d8-436d-af1e-e1077389bac3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:30.271814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eead8-70d8-436d-af1e-e1077389bac3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:30.272091 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f58eead8-70d8-436d-af1e-e1077389bac3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:30.272925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58eead8-70d8-436d-af1e-e1077389bac3 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=075d92d1feca4858ab197d1421621399, user.id=63a7a0e97ec441198ff7c4f9f6604c83) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:30.273237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58eead8-70d8-436d-af1e-e1077389bac3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:30.275266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f58eead8-70d8-436d-af1e-e1077389bac3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:30.293549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 548/1106] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:30 2022] GET /identity/v3/role_assignments?user.id=63a7a0e97ec441198ff7c4f9f6604c83&scope.project.id=075d92d1feca4858ab197d1421621399&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:32.409816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f331754-eb7b-46b3-8592-9f5b215a140f None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.410335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f331754-eb7b-46b3-8592-9f5b215a140f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.410470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f331754-eb7b-46b3-8592-9f5b215a140f None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.411738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 559/1107] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:11:32.417062 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77ce1-0f6d-499a-8ed0-b67c48f210a0 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.417462 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77ce1-0f6d-499a-8ed0-b67c48f210a0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.417650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc77ce1-0f6d-499a-8ed0-b67c48f210a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.434808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7bc77ce1-0f6d-499a-8ed0-b67c48f210a0 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:32.435984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7bc77ce1-0f6d-499a-8ed0-b67c48f210a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:32.456009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:32.501456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 549/1108] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:32.526475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f331754-eb7b-46b3-8592-9f5b215a140f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:32.611000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:32.628823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f331754-eb7b-46b3-8592-9f5b215a140f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:32.662065 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15722343-0cd4-4ac1-9a93-72cd5ae47fa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:32.662841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15722343-0cd4-4ac1-9a93-72cd5ae47fa3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.663124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15722343-0cd4-4ac1-9a93-72cd5ae47fa3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.663401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15722343-0cd4-4ac1-9a93-72cd5ae47fa3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.664604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15722343-0cd4-4ac1-9a93-72cd5ae47fa3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:32.666367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15722343-0cd4-4ac1-9a93-72cd5ae47fa3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:32.667991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 560/1109] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:32.673681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc77ce1-0f6d-499a-8ed0-b67c48f210a0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:32.769948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc77ce1-0f6d-499a-8ed0-b67c48f210a0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:32.802573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c74c5a14-8dfd-4b36-8fac-7a7a4576ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:32.803237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5a14-8dfd-4b36-8fac-7a7a4576ce8e None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:32.803523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5a14-8dfd-4b36-8fac-7a7a4576ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:32.803807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c74c5a14-8dfd-4b36-8fac-7a7a4576ce8e None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:32.804520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c5a14-8dfd-4b36-8fac-7a7a4576ce8e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:32.806704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c74c5a14-8dfd-4b36-8fac-7a7a4576ce8e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:32.826466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 550/1110] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:11:32 2022] POST /identity/v3/users => generated 286 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:11:34.874616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aab5ca-74e8-4817-ae7c-e132d52194d4 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:34.875210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aab5ca-74e8-4817-ae7c-e132d52194d4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:34.875210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aab5ca-74e8-4817-ae7c-e132d52194d4 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:34.876550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 561/1111] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:34.881773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fcb0d-e8f8-4904-bade-465940b2ec72 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:34.882215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fcb0d-e8f8-4904-bade-465940b2ec72 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:34.882318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-700fcb0d-e8f8-4904-bade-465940b2ec72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:34.899142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-700fcb0d-e8f8-4904-bade-465940b2ec72 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:34.900446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-700fcb0d-e8f8-4904-bade-465940b2ec72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:34.907537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:34.966535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 551/1112] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:34.990300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aab5ca-74e8-4817-ae7c-e132d52194d4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.091439 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aab5ca-74e8-4817-ae7c-e132d52194d4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.126554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a74ffd-8931-428c-8b7a-607b12a75dac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.127324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a74ffd-8931-428c-8b7a-607b12a75dac None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.127606 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a74ffd-8931-428c-8b7a-607b12a75dac None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.127883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a74ffd-8931-428c-8b7a-607b12a75dac None admin] PATH_INFO: `/v3/roles/77052d58af4f4b5d9b2d003531462e50` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.129351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a74ffd-8931-428c-8b7a-607b12a75dac None admin] RBAC: Authorizing `identity:get_role(role_id=77052d58af4f4b5d9b2d003531462e50)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.130733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a74ffd-8931-428c-8b7a-607b12a75dac None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.132109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 562/1113] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:34 2022] GET /identity/v3/roles/77052d58af4f4b5d9b2d003531462e50 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:35.138077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fcb0d-e8f8-4904-bade-465940b2ec72 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.235830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-700fcb0d-e8f8-4904-bade-465940b2ec72 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.269622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5984d-899a-4be8-bb8d-185e1a259cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.270493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5984d-899a-4be8-bb8d-185e1a259cff None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.270774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5984d-899a-4be8-bb8d-185e1a259cff None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.271050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de5984d-899a-4be8-bb8d-185e1a259cff None admin] PATH_INFO: `/v3/users/f85be4bcb2734929895140fbe1c02ff6` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.281778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.284530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5984d-899a-4be8-bb8d-185e1a259cff None admin] RBAC: Authorizing `identity:get_user(user_id=f85be4bcb2734929895140fbe1c02ff6)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.286272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0de5984d-899a-4be8-bb8d-185e1a259cff None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.288286 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 552/1114] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/users/f85be4bcb2734929895140fbe1c02ff6 => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:35.294329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a74ffd-8931-428c-8b7a-607b12a75dac None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.317020 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.394011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a74ffd-8931-428c-8b7a-607b12a75dac None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.427315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7433a-b6f4-4777-affe-6abbeb43c911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.427959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7433a-b6f4-4777-affe-6abbeb43c911 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.428244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7433a-b6f4-4777-affe-6abbeb43c911 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.428529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9d7433a-b6f4-4777-affe-6abbeb43c911 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.494400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7433a-b6f4-4777-affe-6abbeb43c911 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.495843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9d7433a-b6f4-4777-affe-6abbeb43c911 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.515370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.594517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 563/1115] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:35.600666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5984d-899a-4be8-bb8d-185e1a259cff None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.698869 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de5984d-899a-4be8-bb8d-185e1a259cff None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.731868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8df65f-3875-4226-9055-efa309e985f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.732725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df65f-3875-4226-9055-efa309e985f4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.733011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df65f-3875-4226-9055-efa309e985f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.733350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8df65f-3875-4226-9055-efa309e985f4 None admin] PATH_INFO: `/v3/projects/075d92d1feca4858ab197d1421621399` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.734480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8df65f-3875-4226-9055-efa309e985f4 None admin] RBAC: Authorizing `identity:get_project(project_id=075d92d1feca4858ab197d1421621399)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.735918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8df65f-3875-4226-9055-efa309e985f4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.737639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 553/1116] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/projects/075d92d1feca4858ab197d1421621399 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:35.744517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7433a-b6f4-4777-affe-6abbeb43c911 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:35.841846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9d7433a-b6f4-4777-affe-6abbeb43c911 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:35.862111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:35.877897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f98f41d-291f-4823-b224-0656b9b128f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:35.878550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98f41d-291f-4823-b224-0656b9b128f1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:35.878832 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98f41d-291f-4823-b224-0656b9b128f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:35.879109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f98f41d-291f-4823-b224-0656b9b128f1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:35.879935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98f41d-291f-4823-b224-0656b9b128f1 None admin] RBAC: Adding query filter params (role.id=77052d58af4f4b5d9b2d003531462e50, scope.project.id=075d92d1feca4858ab197d1421621399, user.id=f85be4bcb2734929895140fbe1c02ff6) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:35.880252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98f41d-291f-4823-b224-0656b9b128f1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:35.881702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f98f41d-291f-4823-b224-0656b9b128f1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:35.887127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 564/1117] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:35 2022] GET /identity/v3/role_assignments?user.id=f85be4bcb2734929895140fbe1c02ff6&scope.project.id=075d92d1feca4858ab197d1421621399&role.id=77052d58af4f4b5d9b2d003531462e50 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:37.889213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd284bf5-799c-4ea8-bea0-047bf6ff1137 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:37.889803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd284bf5-799c-4ea8-bea0-047bf6ff1137 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:37.889803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd284bf5-799c-4ea8-bea0-047bf6ff1137 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:37.891151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 554/1118] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:37.896484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b3891-9257-4c5d-be88-254aef728bf3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:37.896901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b3891-9257-4c5d-be88-254aef728bf3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:37.897038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75b3891-9257-4c5d-be88-254aef728bf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:37.914022 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d75b3891-9257-4c5d-be88-254aef728bf3 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:37.915928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d75b3891-9257-4c5d-be88-254aef728bf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:37.984074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 565/1119] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:38.007041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd284bf5-799c-4ea8-bea0-047bf6ff1137 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:38.081582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:38.115710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd284bf5-799c-4ea8-bea0-047bf6ff1137 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:38.148544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ade74a-8082-4319-a348-0c1d0745a261 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:38.150071 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ade74a-8082-4319-a348-0c1d0745a261 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.150593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ade74a-8082-4319-a348-0c1d0745a261 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.150905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ade74a-8082-4319-a348-0c1d0745a261 None admin] PATH_INFO: `/v3/roles/77052d58af4f4b5d9b2d003531462e50` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.152369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ade74a-8082-4319-a348-0c1d0745a261 None admin] RBAC: Authorizing `identity:get_role(role_id=77052d58af4f4b5d9b2d003531462e50)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:38.153761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ade74a-8082-4319-a348-0c1d0745a261 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:38.155125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 555/1120] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:38 2022] GET /identity/v3/roles/77052d58af4f4b5d9b2d003531462e50 => generated 223 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:38.160968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b3891-9257-4c5d-be88-254aef728bf3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:38.268928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75b3891-9257-4c5d-be88-254aef728bf3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:38.283397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:38.302673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4557bea-d2bd-4f2f-9d65-9475e9d8007b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:38.303522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4557bea-d2bd-4f2f-9d65-9475e9d8007b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.303800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4557bea-d2bd-4f2f-9d65-9475e9d8007b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.304068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4557bea-d2bd-4f2f-9d65-9475e9d8007b None admin] PATH_INFO: `/v3/users/f85be4bcb2734929895140fbe1c02ff6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.305226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4557bea-d2bd-4f2f-9d65-9475e9d8007b None admin] RBAC: Authorizing `identity:get_user(user_id=f85be4bcb2734929895140fbe1c02ff6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:38.306699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4557bea-d2bd-4f2f-9d65-9475e9d8007b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:38.308309 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 566/1121] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:38 2022] GET /identity/v3/users/f85be4bcb2734929895140fbe1c02ff6 => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:38.314107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ade74a-8082-4319-a348-0c1d0745a261 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:38.413437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ade74a-8082-4319-a348-0c1d0745a261 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:38.447264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492706df-0807-432f-b640-85493e7b0ead None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:38.448118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492706df-0807-432f-b640-85493e7b0ead None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.448400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492706df-0807-432f-b640-85493e7b0ead None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.448676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492706df-0807-432f-b640-85493e7b0ead None admin] PATH_INFO: `/v3/projects/075d92d1feca4858ab197d1421621399` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.449880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492706df-0807-432f-b640-85493e7b0ead None admin] RBAC: Authorizing `identity:get_project(project_id=075d92d1feca4858ab197d1421621399)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:38.451354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492706df-0807-432f-b640-85493e7b0ead None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:38.453058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 556/1122] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:38 2022] GET /identity/v3/projects/075d92d1feca4858ab197d1421621399 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:38.459025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4557bea-d2bd-4f2f-9d65-9475e9d8007b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:38.554534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4557bea-d2bd-4f2f-9d65-9475e9d8007b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:38.587984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3190d18d-3157-40e5-9d29-24663b138c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:38.588865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190d18d-3157-40e5-9d29-24663b138c17 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:38.589149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190d18d-3157-40e5-9d29-24663b138c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:38.589425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3190d18d-3157-40e5-9d29-24663b138c17 None admin] PATH_INFO: `/v3/projects/075d92d1feca4858ab197d1421621399/users/f85be4bcb2734929895140fbe1c02ff6/roles/77052d58af4f4b5d9b2d003531462e50` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:38.591628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190d18d-3157-40e5-9d29-24663b138c17 None admin] RBAC: Authorizing `identity:create_grant(project_id=075d92d1feca4858ab197d1421621399, user_id=f85be4bcb2734929895140fbe1c02ff6, role_id=77052d58af4f4b5d9b2d003531462e50)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:38.593076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3190d18d-3157-40e5-9d29-24663b138c17 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:38.606355 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 567/1123] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:38 2022] PUT /identity/v3/projects/075d92d1feca4858ab197d1421621399/users/f85be4bcb2734929895140fbe1c02ff6/roles/77052d58af4f4b5d9b2d003531462e50 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:11:40.608389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7ad81-6a4b-493d-9e4d-227e20c6c3a0 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.608943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7ad81-6a4b-493d-9e4d-227e20c6c3a0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.609158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d7ad81-6a4b-493d-9e4d-227e20c6c3a0 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.610779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 557/1124] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:40.615993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a6a65-16ea-42fc-bd8f-e75bacea8728 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.616404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a6a65-16ea-42fc-bd8f-e75bacea8728 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.616546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946a6a65-16ea-42fc-bd8f-e75bacea8728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.633727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-946a6a65-16ea-42fc-bd8f-e75bacea8728 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:40.634917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-946a6a65-16ea-42fc-bd8f-e75bacea8728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:40.670963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:40.702791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 568/1125] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:40.728081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7ad81-6a4b-493d-9e4d-227e20c6c3a0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.731892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:40.829901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d7ad81-6a4b-493d-9e4d-227e20c6c3a0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:40.863116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c361e-610e-464b-b373-fe0d4fb96c5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:40.863892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c361e-610e-464b-b373-fe0d4fb96c5b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:40.864171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c361e-610e-464b-b373-fe0d4fb96c5b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:40.864442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555c361e-610e-464b-b373-fe0d4fb96c5b None admin] PATH_INFO: `/v3/roles/77052d58af4f4b5d9b2d003531462e50` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:40.865916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c361e-610e-464b-b373-fe0d4fb96c5b None admin] RBAC: Authorizing `identity:get_role(role_id=77052d58af4f4b5d9b2d003531462e50)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:40.867280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555c361e-610e-464b-b373-fe0d4fb96c5b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:40.868833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 558/1126] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:40 2022] GET /identity/v3/roles/77052d58af4f4b5d9b2d003531462e50 => generated 223 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:40.874794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a6a65-16ea-42fc-bd8f-e75bacea8728 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:40.977154 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946a6a65-16ea-42fc-bd8f-e75bacea8728 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:41.010477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89dd45-8ef6-4089-9439-ffd1996afbcb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:41.011340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89dd45-8ef6-4089-9439-ffd1996afbcb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:41.011621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89dd45-8ef6-4089-9439-ffd1996afbcb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:41.011893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e89dd45-8ef6-4089-9439-ffd1996afbcb None admin] PATH_INFO: `/v3/users/f85be4bcb2734929895140fbe1c02ff6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:41.013074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89dd45-8ef6-4089-9439-ffd1996afbcb None admin] RBAC: Authorizing `identity:get_user(user_id=f85be4bcb2734929895140fbe1c02ff6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:41.014663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e89dd45-8ef6-4089-9439-ffd1996afbcb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:41.016288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 569/1127] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:40 2022] GET /identity/v3/users/f85be4bcb2734929895140fbe1c02ff6 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:41.022093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c361e-610e-464b-b373-fe0d4fb96c5b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:41.091379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:41.121896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555c361e-610e-464b-b373-fe0d4fb96c5b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:41.157009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b820-6574-42c5-a33b-384c917824f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:41.157699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b820-6574-42c5-a33b-384c917824f8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:41.157984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b820-6574-42c5-a33b-384c917824f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:41.158259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c0b820-6574-42c5-a33b-384c917824f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:41.222257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0b820-6574-42c5-a33b-384c917824f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:41.223976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c0b820-6574-42c5-a33b-384c917824f8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:41.289498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:41.322618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 559/1128] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:41.328572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89dd45-8ef6-4089-9439-ffd1996afbcb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:41.370944 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:41.428578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e89dd45-8ef6-4089-9439-ffd1996afbcb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:41.462344 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74895c4b-1635-4271-a516-44cf2ea976c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:41.463182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74895c4b-1635-4271-a516-44cf2ea976c5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:41.463468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74895c4b-1635-4271-a516-44cf2ea976c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:41.463748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74895c4b-1635-4271-a516-44cf2ea976c5 None admin] PATH_INFO: `/v3/projects/075d92d1feca4858ab197d1421621399` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:41.464926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74895c4b-1635-4271-a516-44cf2ea976c5 None admin] RBAC: Authorizing `identity:get_project(project_id=075d92d1feca4858ab197d1421621399)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:41.466386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74895c4b-1635-4271-a516-44cf2ea976c5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:41.468083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 570/1129] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:41 2022] GET /identity/v3/projects/075d92d1feca4858ab197d1421621399 => generated 311 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:41.474428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b820-6574-42c5-a33b-384c917824f8 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:41.571952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c0b820-6574-42c5-a33b-384c917824f8 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:41.620165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945191d0-e112-4eba-bde2-0ffc263460d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:41.621293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945191d0-e112-4eba-bde2-0ffc263460d5 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:41.621773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945191d0-e112-4eba-bde2-0ffc263460d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:41.622251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945191d0-e112-4eba-bde2-0ffc263460d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:41.623840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945191d0-e112-4eba-bde2-0ffc263460d5 None admin] RBAC: Adding query filter params (role.id=77052d58af4f4b5d9b2d003531462e50, scope.project.id=075d92d1feca4858ab197d1421621399, user.id=f85be4bcb2734929895140fbe1c02ff6) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:41.624403 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945191d0-e112-4eba-bde2-0ffc263460d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:41.627112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945191d0-e112-4eba-bde2-0ffc263460d5 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:41.637928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 560/1130] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:41 2022] GET /identity/v3/role_assignments?user.id=f85be4bcb2734929895140fbe1c02ff6&scope.project.id=075d92d1feca4858ab197d1421621399&role.id=77052d58af4f4b5d9b2d003531462e50 => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:43.695486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e2812-2e33-40e4-909a-67f02c09285e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.696049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e2812-2e33-40e4-909a-67f02c09285e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.696145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9e2812-2e33-40e4-909a-67f02c09285e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.697527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 571/1131] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:43.702823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf266fe-487c-40aa-8ec9-160206694964 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.703248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf266fe-487c-40aa-8ec9-160206694964 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.703376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf266fe-487c-40aa-8ec9-160206694964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.721325 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6bf266fe-487c-40aa-8ec9-160206694964 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:43.722544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6bf266fe-487c-40aa-8ec9-160206694964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:43.751677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:43.792367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 561/1132] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:43.816111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e2812-2e33-40e4-909a-67f02c09285e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:43.917665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9e2812-2e33-40e4-909a-67f02c09285e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:43.928000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:43.953735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-926f9d3f-dda2-4970-8f6c-c9bd16646991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:43.954647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f9d3f-dda2-4970-8f6c-c9bd16646991 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:43.954931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f9d3f-dda2-4970-8f6c-c9bd16646991 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:43.955207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f9d3f-dda2-4970-8f6c-c9bd16646991 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:43.956380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f9d3f-dda2-4970-8f6c-c9bd16646991 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:43.957855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-926f9d3f-dda2-4970-8f6c-c9bd16646991 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:43.959475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 572/1133] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:11:43.966596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf266fe-487c-40aa-8ec9-160206694964 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:44.071832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bf266fe-487c-40aa-8ec9-160206694964 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:44.105356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e80c51-5e12-42d2-aa7b-ab176c230853 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:44.106035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e80c51-5e12-42d2-aa7b-ab176c230853 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:44.106321 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e80c51-5e12-42d2-aa7b-ab176c230853 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:44.106596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e80c51-5e12-42d2-aa7b-ab176c230853 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:44.107300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e80c51-5e12-42d2-aa7b-ab176c230853 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:44.108735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e80c51-5e12-42d2-aa7b-ab176c230853 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:44.134080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 562/1134] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:11:43 2022] POST /identity/v3/projects => generated 311 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:11:46.218669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe9a8b-d513-43d4-b5fd-dec5b8643b82 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:46.219203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe9a8b-d513-43d4-b5fd-dec5b8643b82 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:46.219203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fe9a8b-d513-43d4-b5fd-dec5b8643b82 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:46.220584 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 573/1135] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:46.226053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c591f1-2e9d-4f0a-bebe-de2476ef5348 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:46.226449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c591f1-2e9d-4f0a-bebe-de2476ef5348 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:46.226606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c591f1-2e9d-4f0a-bebe-de2476ef5348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:46.243875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c0c591f1-2e9d-4f0a-bebe-de2476ef5348 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:46.245038 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c0c591f1-2e9d-4f0a-bebe-de2476ef5348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:46.261832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:46.315814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 563/1136] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:46.341233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe9a8b-d513-43d4-b5fd-dec5b8643b82 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:46.440141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fe9a8b-d513-43d4-b5fd-dec5b8643b82 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:46.473980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48ff020-6e58-479f-b33a-1d91efabef47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:46.474695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ff020-6e58-479f-b33a-1d91efabef47 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:46.474973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ff020-6e58-479f-b33a-1d91efabef47 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:46.475242 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48ff020-6e58-479f-b33a-1d91efabef47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:46.476399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ff020-6e58-479f-b33a-1d91efabef47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:46.477992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48ff020-6e58-479f-b33a-1d91efabef47 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:46.479595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 574/1137] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:11:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:46.485429 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c591f1-2e9d-4f0a-bebe-de2476ef5348 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:46.581984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c591f1-2e9d-4f0a-bebe-de2476ef5348 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:46.615445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ec7021-dc14-4363-893c-103973005f1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:46.616108 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec7021-dc14-4363-893c-103973005f1b None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:46.616391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec7021-dc14-4363-893c-103973005f1b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:46.616666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ec7021-dc14-4363-893c-103973005f1b None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:46.617376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec7021-dc14-4363-893c-103973005f1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:46.618954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ec7021-dc14-4363-893c-103973005f1b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:46.639329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 564/1138] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:11:46 2022] POST /identity/v3/users => generated 286 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:11:48.733492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaadf5-aaa5-4805-abcd-aedd42eb4e96 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.734074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaadf5-aaa5-4805-abcd-aedd42eb4e96 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.734074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaadf5-aaa5-4805-abcd-aedd42eb4e96 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.735444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 575/1139] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:48.740599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de663566-1af3-43c9-a0d2-bd6df182c31b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.741028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de663566-1af3-43c9-a0d2-bd6df182c31b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.741163 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de663566-1af3-43c9-a0d2-bd6df182c31b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:48.747947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.759268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-de663566-1af3-43c9-a0d2-bd6df182c31b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:48.760451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-de663566-1af3-43c9-a0d2-bd6df182c31b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:48.826201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 565/1140] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:48.849663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaadf5-aaa5-4805-abcd-aedd42eb4e96 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:48.897772 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:48.953076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaadf5-aaa5-4805-abcd-aedd42eb4e96 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:48.991125 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4702d22-36d1-42ff-99b4-4be5784ff8b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:48.991831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4702d22-36d1-42ff-99b4-4be5784ff8b7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:48.992141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4702d22-36d1-42ff-99b4-4be5784ff8b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:48.992420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4702d22-36d1-42ff-99b4-4be5784ff8b7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:49.003374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4702d22-36d1-42ff-99b4-4be5784ff8b7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b4702d22-36d1-42ff-99b4-4be5784ff8b7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:49.011000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:49.012188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:49.013568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 576/1141] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:49.018300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de663566-1af3-43c9-a0d2-bd6df182c31b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:49.120146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de663566-1af3-43c9-a0d2-bd6df182c31b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:49.155933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12694dbe-6407-4427-8357-1b3fd55a2b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:49.156678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12694dbe-6407-4427-8357-1b3fd55a2b2e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:49.157059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12694dbe-6407-4427-8357-1b3fd55a2b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:49.157385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12694dbe-6407-4427-8357-1b3fd55a2b2e None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:49.158220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12694dbe-6407-4427-8357-1b3fd55a2b2e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:49.158582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12694dbe-6407-4427-8357-1b3fd55a2b2e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:49.160003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12694dbe-6407-4427-8357-1b3fd55a2b2e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:49.161437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:49.170165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 566/1142] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:49.175917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4702d22-36d1-42ff-99b4-4be5784ff8b7 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:49.276980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4702d22-36d1-42ff-99b4-4be5784ff8b7 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:49.279561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:49.320536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532e344-baa5-42d9-bf75-3062f566a81c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:49.322041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532e344-baa5-42d9-bf75-3062f566a81c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:49.322330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532e344-baa5-42d9-bf75-3062f566a81c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:49.322611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532e344-baa5-42d9-bf75-3062f566a81c None admin] PATH_INFO: `/v3/users/d065d8cd83e946b7913656880a84a113` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:49.336095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532e344-baa5-42d9-bf75-3062f566a81c None admin] RBAC: Authorizing `identity:get_user(user_id=d065d8cd83e946b7913656880a84a113)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:49.337630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532e344-baa5-42d9-bf75-3062f566a81c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:49.339303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 577/1143] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:49 2022] GET /identity/v3/users/d065d8cd83e946b7913656880a84a113 => generated 286 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:49.345056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12694dbe-6407-4427-8357-1b3fd55a2b2e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:49.448714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12694dbe-6407-4427-8357-1b3fd55a2b2e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:49.483696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c81461-9998-4cf5-8516-41ae5bac80df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:49.484189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c81461-9998-4cf5-8516-41ae5bac80df None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:49.484476 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c81461-9998-4cf5-8516-41ae5bac80df None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:49.484810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c81461-9998-4cf5-8516-41ae5bac80df None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:49.544870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:49.551310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c81461-9998-4cf5-8516-41ae5bac80df None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:49.552767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c81461-9998-4cf5-8516-41ae5bac80df None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:49.648628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 567/1144] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:49.654654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532e344-baa5-42d9-bf75-3062f566a81c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:49.753363 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532e344-baa5-42d9-bf75-3062f566a81c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:49.787372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335c7618-3eed-4f30-9e33-2662a0e33b2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:49.788213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c7618-3eed-4f30-9e33-2662a0e33b2c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:49.788493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c7618-3eed-4f30-9e33-2662a0e33b2c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:49.788764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335c7618-3eed-4f30-9e33-2662a0e33b2c None admin] PATH_INFO: `/v3/projects/9108d557578c40b383808da93f539fd6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:49.790567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c7618-3eed-4f30-9e33-2662a0e33b2c None admin] RBAC: Authorizing `identity:get_project(project_id=9108d557578c40b383808da93f539fd6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:49.791957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335c7618-3eed-4f30-9e33-2662a0e33b2c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:49.793676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 578/1145] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:49 2022] GET /identity/v3/projects/9108d557578c40b383808da93f539fd6 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:49.799891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c81461-9998-4cf5-8516-41ae5bac80df None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:49.892935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:49.899468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c81461-9998-4cf5-8516-41ae5bac80df None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:49.933826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baab708e-85eb-4c76-96aa-5ea5523a4d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:49.934474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab708e-85eb-4c76-96aa-5ea5523a4d1b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:49.934761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab708e-85eb-4c76-96aa-5ea5523a4d1b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:49.935037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baab708e-85eb-4c76-96aa-5ea5523a4d1b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:49.935855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab708e-85eb-4c76-96aa-5ea5523a4d1b None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=9108d557578c40b383808da93f539fd6, user.id=d065d8cd83e946b7913656880a84a113) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:49.936208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab708e-85eb-4c76-96aa-5ea5523a4d1b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:49.937666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baab708e-85eb-4c76-96aa-5ea5523a4d1b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:49.943086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 568/1146] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:49 2022] GET /identity/v3/role_assignments?user.id=d065d8cd83e946b7913656880a84a113&scope.project.id=9108d557578c40b383808da93f539fd6&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 259 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:51.955982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c32884-4422-4efb-8e89-9d61b4501381 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.956562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c32884-4422-4efb-8e89-9d61b4501381 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.956562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c32884-4422-4efb-8e89-9d61b4501381 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.957920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 579/1147] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:51.963821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36232e5c-1a0d-40d0-bab0-783380b4737b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:51.964289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36232e5c-1a0d-40d0-bab0-783380b4737b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:51.964391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36232e5c-1a0d-40d0-bab0-783380b4737b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:51.981863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-36232e5c-1a0d-40d0-bab0-783380b4737b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:51.983055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-36232e5c-1a0d-40d0-bab0-783380b4737b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:52.049007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 569/1148] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:11:52.071706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c32884-4422-4efb-8e89-9d61b4501381 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:52.103218 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:52.183232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c32884-4422-4efb-8e89-9d61b4501381 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.218765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e51da-773f-44b6-acf5-80e3879699ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.219477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e51da-773f-44b6-acf5-80e3879699ff None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.219759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e51da-773f-44b6-acf5-80e3879699ff None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.220032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3e51da-773f-44b6-acf5-80e3879699ff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.231645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3e51da-773f-44b6-acf5-80e3879699ff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-1b3e51da-773f-44b6-acf5-80e3879699ff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:52.239210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:52.240334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:52.241215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 580/1149] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:52.247144 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36232e5c-1a0d-40d0-bab0-783380b4737b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:52.338196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:52.348341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36232e5c-1a0d-40d0-bab0-783380b4737b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.382680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714ffb-4b55-472b-9ac2-0c9d541e3319 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.383347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714ffb-4b55-472b-9ac2-0c9d541e3319 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.383628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714ffb-4b55-472b-9ac2-0c9d541e3319 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.383898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d714ffb-4b55-472b-9ac2-0c9d541e3319 None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.384635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d714ffb-4b55-472b-9ac2-0c9d541e3319 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:52.384935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d714ffb-4b55-472b-9ac2-0c9d541e3319 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:52.386387 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d714ffb-4b55-472b-9ac2-0c9d541e3319 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.394862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 570/1150] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:52.401016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e51da-773f-44b6-acf5-80e3879699ff None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:52.498843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:52.509556 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3e51da-773f-44b6-acf5-80e3879699ff None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.544790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4b7c4-9bd1-4fbf-b78d-7c97a3b3cc58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.545687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4b7c4-9bd1-4fbf-b78d-7c97a3b3cc58 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.545973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4b7c4-9bd1-4fbf-b78d-7c97a3b3cc58 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.546250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc4b7c4-9bd1-4fbf-b78d-7c97a3b3cc58 None admin] PATH_INFO: `/v3/users/d065d8cd83e946b7913656880a84a113` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.547445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc4b7c4-9bd1-4fbf-b78d-7c97a3b3cc58 None admin] RBAC: Authorizing `identity:get_user(user_id=d065d8cd83e946b7913656880a84a113)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:52.548868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc4b7c4-9bd1-4fbf-b78d-7c97a3b3cc58 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.550632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 581/1151] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:52 2022] GET /identity/v3/users/d065d8cd83e946b7913656880a84a113 => generated 286 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:52.558246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714ffb-4b55-472b-9ac2-0c9d541e3319 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:52.677660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d714ffb-4b55-472b-9ac2-0c9d541e3319 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.712939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7267cb-599a-40ff-bdc6-d922f46f4ef2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.713837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7267cb-599a-40ff-bdc6-d922f46f4ef2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.714126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7267cb-599a-40ff-bdc6-d922f46f4ef2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.714405 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7267cb-599a-40ff-bdc6-d922f46f4ef2 None admin] PATH_INFO: `/v3/projects/9108d557578c40b383808da93f539fd6` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.715599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7267cb-599a-40ff-bdc6-d922f46f4ef2 None admin] RBAC: Authorizing `identity:get_project(project_id=9108d557578c40b383808da93f539fd6)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:52.717053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f7267cb-599a-40ff-bdc6-d922f46f4ef2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.718827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 571/1152] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:52 2022] GET /identity/v3/projects/9108d557578c40b383808da93f539fd6 => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:52.725293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4b7c4-9bd1-4fbf-b78d-7c97a3b3cc58 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:52.827264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc4b7c4-9bd1-4fbf-b78d-7c97a3b3cc58 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:52.861276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e310f50-0f96-4451-b8c8-901845387359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:52.862186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e310f50-0f96-4451-b8c8-901845387359 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:52.862471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e310f50-0f96-4451-b8c8-901845387359 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:52.862748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e310f50-0f96-4451-b8c8-901845387359 None admin] PATH_INFO: `/v3/projects/9108d557578c40b383808da93f539fd6/users/d065d8cd83e946b7913656880a84a113/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:52.864676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e310f50-0f96-4451-b8c8-901845387359 None admin] RBAC: Authorizing `identity:create_grant(project_id=9108d557578c40b383808da93f539fd6, user_id=d065d8cd83e946b7913656880a84a113, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:52.866137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e310f50-0f96-4451-b8c8-901845387359 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:52.879600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 582/1153] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:11:52 2022] PUT /identity/v3/projects/9108d557578c40b383808da93f539fd6/users/d065d8cd83e946b7913656880a84a113/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:11:54.924376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594543c-327c-4095-a60e-9fbc1c859263 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:54.924911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594543c-327c-4095-a60e-9fbc1c859263 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:54.924911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0594543c-327c-4095-a60e-9fbc1c859263 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:54.926342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 572/1154] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:54.932761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eedc7-febe-457a-92c0-1831433b3bbe None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:54.933201 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eedc7-febe-457a-92c0-1831433b3bbe None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:54.933710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55eedc7-febe-457a-92c0-1831433b3bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:54.952169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a55eedc7-febe-457a-92c0-1831433b3bbe None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:54.953462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a55eedc7-febe-457a-92c0-1831433b3bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:54.967361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:55.025315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 583/1155] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:55.050779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594543c-327c-4095-a60e-9fbc1c859263 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.075147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:55.159208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0594543c-327c-4095-a60e-9fbc1c859263 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.193866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c63e4-ed99-4acf-883b-db5aee7abcff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.194566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c63e4-ed99-4acf-883b-db5aee7abcff None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.194848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c63e4-ed99-4acf-883b-db5aee7abcff None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.195124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-736c63e4-ed99-4acf-883b-db5aee7abcff None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.206188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-736c63e4-ed99-4acf-883b-db5aee7abcff None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-736c63e4-ed99-4acf-883b-db5aee7abcff None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:11:55.214796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:55.216022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:11:55.217026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 573/1156] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:55.222245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55eedc7-febe-457a-92c0-1831433b3bbe None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.327484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55eedc7-febe-457a-92c0-1831433b3bbe None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.365024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c0c56-4b94-412b-9f0c-c5466be4b31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.365719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c0c56-4b94-412b-9f0c-c5466be4b31e None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.366008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c0c56-4b94-412b-9f0c-c5466be4b31e None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.366288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c0c56-4b94-412b-9f0c-c5466be4b31e None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.367034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c0c56-4b94-412b-9f0c-c5466be4b31e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:55.367987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c0c56-4b94-412b-9f0c-c5466be4b31e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:55.369580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5c0c56-4b94-412b-9f0c-c5466be4b31e None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:55.378187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 584/1157] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:55.384885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c63e4-ed99-4acf-883b-db5aee7abcff None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.464054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:55.483960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-736c63e4-ed99-4acf-883b-db5aee7abcff None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.520444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587094-60a9-44d7-a40a-ebcce47adb69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.521309 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587094-60a9-44d7-a40a-ebcce47adb69 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.521617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587094-60a9-44d7-a40a-ebcce47adb69 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.521895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52587094-60a9-44d7-a40a-ebcce47adb69 None admin] PATH_INFO: `/v3/users/d065d8cd83e946b7913656880a84a113` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.523054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52587094-60a9-44d7-a40a-ebcce47adb69 None admin] RBAC: Authorizing `identity:get_user(user_id=d065d8cd83e946b7913656880a84a113)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:55.524479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52587094-60a9-44d7-a40a-ebcce47adb69 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:55.526134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 574/1158] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/users/d065d8cd83e946b7913656880a84a113 => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:55.532081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c0c56-4b94-412b-9f0c-c5466be4b31e None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.545299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:55.635127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c0c56-4b94-412b-9f0c-c5466be4b31e None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.670299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535d560-b9f0-4f8b-81a5-34fb34588268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.670946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d560-b9f0-4f8b-81a5-34fb34588268 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.671229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d560-b9f0-4f8b-81a5-34fb34588268 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.671538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0535d560-b9f0-4f8b-81a5-34fb34588268 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.741840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535d560-b9f0-4f8b-81a5-34fb34588268 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:55.743235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0535d560-b9f0-4f8b-81a5-34fb34588268 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:55.754381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:55.841807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 585/1159] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:55.847985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587094-60a9-44d7-a40a-ebcce47adb69 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:55.944278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52587094-60a9-44d7-a40a-ebcce47adb69 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:55.978641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd15c86-54b3-4c12-94c6-3310ae82f370 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:55.979491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd15c86-54b3-4c12-94c6-3310ae82f370 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:55.979771 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd15c86-54b3-4c12-94c6-3310ae82f370 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:55.980043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd15c86-54b3-4c12-94c6-3310ae82f370 None admin] PATH_INFO: `/v3/projects/9108d557578c40b383808da93f539fd6` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:55.981198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd15c86-54b3-4c12-94c6-3310ae82f370 None admin] RBAC: Authorizing `identity:get_project(project_id=9108d557578c40b383808da93f539fd6)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:55.982664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd15c86-54b3-4c12-94c6-3310ae82f370 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:55.984348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 575/1160] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/projects/9108d557578c40b383808da93f539fd6 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:55.990497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535d560-b9f0-4f8b-81a5-34fb34588268 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:56.087946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0535d560-b9f0-4f8b-81a5-34fb34588268 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:56.098674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:56.122510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03640af5-f61d-4b5d-9775-838117a8aa39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:56.123159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03640af5-f61d-4b5d-9775-838117a8aa39 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:56.123443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03640af5-f61d-4b5d-9775-838117a8aa39 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:56.123721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03640af5-f61d-4b5d-9775-838117a8aa39 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:56.124537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03640af5-f61d-4b5d-9775-838117a8aa39 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=9108d557578c40b383808da93f539fd6, user.id=d065d8cd83e946b7913656880a84a113) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:56.124849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03640af5-f61d-4b5d-9775-838117a8aa39 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:56.126299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03640af5-f61d-4b5d-9775-838117a8aa39 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:56.131988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 586/1161] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:11:55 2022] GET /identity/v3/role_assignments?user.id=d065d8cd83e946b7913656880a84a113&scope.project.id=9108d557578c40b383808da93f539fd6&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:11:58.271558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32509b6-5c23-47b0-b33b-f9ed4d3b39a0 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:58.272129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32509b6-5c23-47b0-b33b-f9ed4d3b39a0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:58.272129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32509b6-5c23-47b0-b33b-f9ed4d3b39a0 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:58.274002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 576/1162] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:11:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:11:58.279543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73e65-1b21-484a-a790-0dde67afa56a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:58.280104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73e65-1b21-484a-a790-0dde67afa56a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:58.280104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed73e65-1b21-484a-a790-0dde67afa56a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:58.297845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3ed73e65-1b21-484a-a790-0dde67afa56a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:11:58.299046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3ed73e65-1b21-484a-a790-0dde67afa56a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:11:58.367432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 587/1163] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:11:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:11:58.390992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32509b6-5c23-47b0-b33b-f9ed4d3b39a0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:58.410321 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:58.498807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32509b6-5c23-47b0-b33b-f9ed4d3b39a0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:58.535629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05b7d7-06b8-481f-87a1-e9b719970f98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:58.536196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05b7d7-06b8-481f-87a1-e9b719970f98 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:58.536479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05b7d7-06b8-481f-87a1-e9b719970f98 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:58.536757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f05b7d7-06b8-481f-87a1-e9b719970f98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:58.607490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05b7d7-06b8-481f-87a1-e9b719970f98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:58.608958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f05b7d7-06b8-481f-87a1-e9b719970f98 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:58.626596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:58.721599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 577/1164] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:11:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:11:58.728149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed73e65-1b21-484a-a790-0dde67afa56a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:58.835060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed73e65-1b21-484a-a790-0dde67afa56a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:58.837526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:58.871306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4a4e9c-6b1a-412c-a7e2-18b8dc214126 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:58.872095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a4e9c-6b1a-412c-a7e2-18b8dc214126 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:58.872382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a4e9c-6b1a-412c-a7e2-18b8dc214126 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:58.872664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4a4e9c-6b1a-412c-a7e2-18b8dc214126 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:58.879928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a4e9c-6b1a-412c-a7e2-18b8dc214126 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:58.881397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4a4e9c-6b1a-412c-a7e2-18b8dc214126 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-5a4a4e9c-6b1a-412c-a7e2-18b8dc214126 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:11:58.897082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:11:58.898479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:11:58.899502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 588/1165] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Jul 20 19:11:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:11:58.905293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05b7d7-06b8-481f-87a1-e9b719970f98 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:11:59.003741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f05b7d7-06b8-481f-87a1-e9b719970f98 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:11:59.020725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:11:59.038661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc3ad127-085a-4f3a-bf65-1494aee5ddd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:11:59.039443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ad127-085a-4f3a-bf65-1494aee5ddd1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:11:59.039730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ad127-085a-4f3a-bf65-1494aee5ddd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:11:59.040010 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc3ad127-085a-4f3a-bf65-1494aee5ddd1 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:11:59.040732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ad127-085a-4f3a-bf65-1494aee5ddd1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:11:59.041039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ad127-085a-4f3a-bf65-1494aee5ddd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:11:59.042449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc3ad127-085a-4f3a-bf65-1494aee5ddd1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:11:59.049934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 578/1166] 10.222.0.69 () {56 vars in 1115 bytes} [Wed Jul 20 19:11:58 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:12:01.112979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b023c-24b9-44c1-b8e0-dd6a91530383 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:01.113546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b023c-24b9-44c1-b8e0-dd6a91530383 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:01.113646 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3b023c-24b9-44c1-b8e0-dd6a91530383 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:01.114967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 589/1167] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:12:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:01.120738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c91b1a-5b0c-496e-b089-cbfedd176f2b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:01.121144 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c91b1a-5b0c-496e-b089-cbfedd176f2b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:01.121303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c91b1a-5b0c-496e-b089-cbfedd176f2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:01.137957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-02c91b1a-5b0c-496e-b089-cbfedd176f2b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:01.139137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-02c91b1a-5b0c-496e-b089-cbfedd176f2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:01.204349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 579/1168] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:01.229008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b023c-24b9-44c1-b8e0-dd6a91530383 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:01.331372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3b023c-24b9-44c1-b8e0-dd6a91530383 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:01.369488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b43f9a-1e5a-4eaa-95c9-1455d9e565c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:01.370096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b43f9a-1e5a-4eaa-95c9-1455d9e565c1 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:01.370376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b43f9a-1e5a-4eaa-95c9-1455d9e565c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:01.370650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b43f9a-1e5a-4eaa-95c9-1455d9e565c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:01.371392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b43f9a-1e5a-4eaa-95c9-1455d9e565c1 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:01.372757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b43f9a-1e5a-4eaa-95c9-1455d9e565c1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:01.388608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 590/1169] 10.222.0.69 () {60 vars in 1138 bytes} [Wed Jul 20 19:12:01 2022] POST /identity/v3/domains => generated 267 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:12:03.462842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc242942-0da9-47ab-97c5-bb1472558956 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:03.463398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc242942-0da9-47ab-97c5-bb1472558956 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:03.463398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc242942-0da9-47ab-97c5-bb1472558956 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:03.464776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 580/1170] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:12:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:03.470159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7ba4-8f64-45a0-b3c7-61b01a7e533d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:03.470618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7ba4-8f64-45a0-b3c7-61b01a7e533d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:03.470726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab9c7ba4-8f64-45a0-b3c7-61b01a7e533d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:03.488245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ab9c7ba4-8f64-45a0-b3c7-61b01a7e533d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:03.489438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ab9c7ba4-8f64-45a0-b3c7-61b01a7e533d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:03.507119 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:03.557933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 591/1171] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:03.582328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc242942-0da9-47ab-97c5-bb1472558956 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:03.683854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc242942-0da9-47ab-97c5-bb1472558956 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:03.695439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:03.719010 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427ebbb2-4935-4bbf-a989-b768e946fb2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:03.719872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ebbb2-4935-4bbf-a989-b768e946fb2d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:03.720153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ebbb2-4935-4bbf-a989-b768e946fb2d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:03.720426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427ebbb2-4935-4bbf-a989-b768e946fb2d None admin] PATH_INFO: `/v3/domains/f89c7c5b9fd5446194ce717bdccf9a70` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:03.731818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ebbb2-4935-4bbf-a989-b768e946fb2d None admin] RBAC: Authorizing `identity:get_domain(domain_id=f89c7c5b9fd5446194ce717bdccf9a70)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:03.733270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427ebbb2-4935-4bbf-a989-b768e946fb2d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:03.735215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 581/1172] 10.222.0.69 () {56 vars in 1183 bytes} [Wed Jul 20 19:12:03 2022] GET /identity/v3/domains/f89c7c5b9fd5446194ce717bdccf9a70 => generated 267 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:03.740947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c7ba4-8f64-45a0-b3c7-61b01a7e533d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:03.840602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab9c7ba4-8f64-45a0-b3c7-61b01a7e533d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:03.875409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4ac6d9-0cb8-4b5a-9041-03ebc8680755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:03.876065 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ac6d9-0cb8-4b5a-9041-03ebc8680755 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:03.876349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ac6d9-0cb8-4b5a-9041-03ebc8680755 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:03.876627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4ac6d9-0cb8-4b5a-9041-03ebc8680755 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:03.877354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ac6d9-0cb8-4b5a-9041-03ebc8680755 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:03.878825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4ac6d9-0cb8-4b5a-9041-03ebc8680755 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:03.903811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 592/1173] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:12:03 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:12:05.985095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb649e7-d4f8-4136-ae2e-9f84958de9b0 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:05.985689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb649e7-d4f8-4136-ae2e-9f84958de9b0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:05.985923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb649e7-d4f8-4136-ae2e-9f84958de9b0 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:05.987330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 582/1174] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:05.993164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377ad8-c566-493f-b5a5-84d3dfd97796 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:05.993671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377ad8-c566-493f-b5a5-84d3dfd97796 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:05.993959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03377ad8-c566-493f-b5a5-84d3dfd97796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:06.005557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:06.011446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-03377ad8-c566-493f-b5a5-84d3dfd97796 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:06.012998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-03377ad8-c566-493f-b5a5-84d3dfd97796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:06.079770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 593/1175] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:06.105290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb649e7-d4f8-4136-ae2e-9f84958de9b0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:06.204264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb649e7-d4f8-4136-ae2e-9f84958de9b0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:06.237756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-590413ad-755d-45c8-b2d6-36ca91bbee27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:06.238530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590413ad-755d-45c8-b2d6-36ca91bbee27 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:06.238806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590413ad-755d-45c8-b2d6-36ca91bbee27 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:06.239074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-590413ad-755d-45c8-b2d6-36ca91bbee27 None admin] PATH_INFO: `/v3/domains/f89c7c5b9fd5446194ce717bdccf9a70` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:06.240236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590413ad-755d-45c8-b2d6-36ca91bbee27 None admin] RBAC: Authorizing `identity:get_domain(domain_id=f89c7c5b9fd5446194ce717bdccf9a70)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:06.241856 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-590413ad-755d-45c8-b2d6-36ca91bbee27 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:06.243540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 583/1176] 10.222.0.69 () {56 vars in 1183 bytes} [Wed Jul 20 19:12:06 2022] GET /identity/v3/domains/f89c7c5b9fd5446194ce717bdccf9a70 => generated 267 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:06.250176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03377ad8-c566-493f-b5a5-84d3dfd97796 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:06.350118 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03377ad8-c566-493f-b5a5-84d3dfd97796 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:06.385008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89df9e99-bce1-4c0a-bf67-2a0e32259d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:06.385697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df9e99-bce1-4c0a-bf67-2a0e32259d9c None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:06.385981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df9e99-bce1-4c0a-bf67-2a0e32259d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:06.386252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89df9e99-bce1-4c0a-bf67-2a0e32259d9c None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:06.386952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df9e99-bce1-4c0a-bf67-2a0e32259d9c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:06.388373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89df9e99-bce1-4c0a-bf67-2a0e32259d9c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:06.396069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:06.409583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 594/1177] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:12:06 2022] POST /identity/v3/users => generated 311 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:12:08.512455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294ca21-1347-4bf6-8a19-068cfcb77bf5 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:08.513027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294ca21-1347-4bf6-8a19-068cfcb77bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:08.513027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1294ca21-1347-4bf6-8a19-068cfcb77bf5 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:08.514498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 584/1178] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:12:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:08.520014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9d90-9d88-491c-8208-4d1529a89635 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:08.520513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9d90-9d88-491c-8208-4d1529a89635 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:08.520617 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dba9d90-9d88-491c-8208-4d1529a89635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:08.537686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6dba9d90-9d88-491c-8208-4d1529a89635 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:08.538964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6dba9d90-9d88-491c-8208-4d1529a89635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:08.606784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 595/1179] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:08.631121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294ca21-1347-4bf6-8a19-068cfcb77bf5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:08.672368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:08.733824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1294ca21-1347-4bf6-8a19-068cfcb77bf5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:08.767574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c68fe-7346-4187-a0e4-b9ef0cd5d776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:08.768364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c68fe-7346-4187-a0e4-b9ef0cd5d776 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:08.768646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c68fe-7346-4187-a0e4-b9ef0cd5d776 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:08.768919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2c68fe-7346-4187-a0e4-b9ef0cd5d776 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:08.780195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2c68fe-7346-4187-a0e4-b9ef0cd5d776 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1c2c68fe-7346-4187-a0e4-b9ef0cd5d776 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:12:08.787750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:08.789149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:12:08.790224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 585/1180] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:12:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:08.795288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba9d90-9d88-491c-8208-4d1529a89635 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:08.893194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dba9d90-9d88-491c-8208-4d1529a89635 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:08.916403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:08.928484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6e9c9-176d-40d1-be83-0bbf65ce14f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:08.929841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e9c9-176d-40d1-be83-0bbf65ce14f8 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:08.930573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e9c9-176d-40d1-be83-0bbf65ce14f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:08.930573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ea6e9c9-176d-40d1-be83-0bbf65ce14f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:08.931126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6e9c9-176d-40d1-be83-0bbf65ce14f8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:08.931514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6e9c9-176d-40d1-be83-0bbf65ce14f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:08.932888 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ea6e9c9-176d-40d1-be83-0bbf65ce14f8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:08.941103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 596/1181] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:12:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:08.947214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c68fe-7346-4187-a0e4-b9ef0cd5d776 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:09.042748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:09.046730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2c68fe-7346-4187-a0e4-b9ef0cd5d776 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:09.080589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c925d-d960-4e1a-8a25-7900252304b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:09.081463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c925d-d960-4e1a-8a25-7900252304b6 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:09.081782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c925d-d960-4e1a-8a25-7900252304b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:09.082063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c925d-d960-4e1a-8a25-7900252304b6 None admin] PATH_INFO: `/v3/users/6eafabdd3d154b909564ff40e7393e5d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:09.095507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c925d-d960-4e1a-8a25-7900252304b6 None admin] RBAC: Authorizing `identity:get_user(user_id=6eafabdd3d154b909564ff40e7393e5d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:09.096969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c925d-d960-4e1a-8a25-7900252304b6 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:09.099002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 586/1182] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:08 2022] GET /identity/v3/users/6eafabdd3d154b909564ff40e7393e5d => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:09.105561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6e9c9-176d-40d1-be83-0bbf65ce14f8 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:09.206134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ea6e9c9-176d-40d1-be83-0bbf65ce14f8 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:09.239368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99627d80-54ec-41b4-81ea-7972ba2e0910 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:09.240014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99627d80-54ec-41b4-81ea-7972ba2e0910 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:09.240294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99627d80-54ec-41b4-81ea-7972ba2e0910 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:09.240565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99627d80-54ec-41b4-81ea-7972ba2e0910 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:09.293130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:09.309291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99627d80-54ec-41b4-81ea-7972ba2e0910 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:09.310809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99627d80-54ec-41b4-81ea-7972ba2e0910 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:09.410730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 597/1183] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:12:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:12:09.416714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c925d-d960-4e1a-8a25-7900252304b6 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:09.513595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c925d-d960-4e1a-8a25-7900252304b6 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:09.545921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcdcda7-9eb2-46de-8b4d-a01a5dccaa7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:09.546763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdcda7-9eb2-46de-8b4d-a01a5dccaa7f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:09.547048 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdcda7-9eb2-46de-8b4d-a01a5dccaa7f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:09.547322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcdcda7-9eb2-46de-8b4d-a01a5dccaa7f None admin] PATH_INFO: `/v3/projects/d70c95a3987f429e86092940352f8c6e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:09.548489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdcda7-9eb2-46de-8b4d-a01a5dccaa7f None admin] RBAC: Authorizing `identity:get_project(project_id=d70c95a3987f429e86092940352f8c6e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:09.549967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcdcda7-9eb2-46de-8b4d-a01a5dccaa7f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:09.551673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 587/1184] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:09 2022] GET /identity/v3/projects/d70c95a3987f429e86092940352f8c6e => generated 361 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:09.558900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99627d80-54ec-41b4-81ea-7972ba2e0910 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:09.644031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:09.666908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99627d80-54ec-41b4-81ea-7972ba2e0910 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:09.700051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84206089-71a7-4567-bb3c-9b6f22825e8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:09.700691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84206089-71a7-4567-bb3c-9b6f22825e8f None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:09.700970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84206089-71a7-4567-bb3c-9b6f22825e8f None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:09.701238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84206089-71a7-4567-bb3c-9b6f22825e8f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:09.702076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84206089-71a7-4567-bb3c-9b6f22825e8f None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=d70c95a3987f429e86092940352f8c6e, user.id=6eafabdd3d154b909564ff40e7393e5d) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:09.702385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84206089-71a7-4567-bb3c-9b6f22825e8f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:09.703793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84206089-71a7-4567-bb3c-9b6f22825e8f None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:09.709027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 598/1185] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:12:09 2022] GET /identity/v3/role_assignments?user.id=6eafabdd3d154b909564ff40e7393e5d&scope.project.id=d70c95a3987f429e86092940352f8c6e&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 259 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:11.722194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b12ef-7910-46b4-a6d9-cbb1220185f1 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:11.723020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b12ef-7910-46b4-a6d9-cbb1220185f1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:11.723020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278b12ef-7910-46b4-a6d9-cbb1220185f1 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:11.724386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 588/1186] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:12:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:11.729884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cad65-0a02-4566-a2fc-7aae7894d4f1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:11.730274 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cad65-0a02-4566-a2fc-7aae7894d4f1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:11.730445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7cad65-0a02-4566-a2fc-7aae7894d4f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:11.747430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2f7cad65-0a02-4566-a2fc-7aae7894d4f1 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:11.748616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2f7cad65-0a02-4566-a2fc-7aae7894d4f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:11.814057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 599/1187] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:12:11.837007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b12ef-7910-46b4-a6d9-cbb1220185f1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:11.862187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:11.936838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278b12ef-7910-46b4-a6d9-cbb1220185f1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:11.969868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9a935-6198-4be1-abca-861b9597559b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:11.970641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9a935-6198-4be1-abca-861b9597559b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:11.970925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9a935-6198-4be1-abca-861b9597559b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:11.971196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a9a935-6198-4be1-abca-861b9597559b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:11.982222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a9a935-6198-4be1-abca-861b9597559b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-24a9a935-6198-4be1-abca-861b9597559b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:12:11.989573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:11.990672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:12:11.992504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 589/1188] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:12:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:11.997572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cad65-0a02-4566-a2fc-7aae7894d4f1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:12.073534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:12.094078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7cad65-0a02-4566-a2fc-7aae7894d4f1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:12.127886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dedb1-47e2-4436-bfe7-5673dcbc34cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:12.128539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dedb1-47e2-4436-bfe7-5673dcbc34cc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:12.128819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dedb1-47e2-4436-bfe7-5673dcbc34cc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:12.129093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dedb1-47e2-4436-bfe7-5673dcbc34cc None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:12.129848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dedb1-47e2-4436-bfe7-5673dcbc34cc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:12.130153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dedb1-47e2-4436-bfe7-5673dcbc34cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:12.131504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dedb1-47e2-4436-bfe7-5673dcbc34cc None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:12.139427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 600/1189] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:12:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:12.145218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9a935-6198-4be1-abca-861b9597559b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:12.228676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:12.244140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a9a935-6198-4be1-abca-861b9597559b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:12.276239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6770509-8424-4b82-803e-462c5d9f4887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:12.277089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6770509-8424-4b82-803e-462c5d9f4887 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:12.277365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6770509-8424-4b82-803e-462c5d9f4887 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:12.277670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6770509-8424-4b82-803e-462c5d9f4887 None admin] PATH_INFO: `/v3/users/6eafabdd3d154b909564ff40e7393e5d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:12.278839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6770509-8424-4b82-803e-462c5d9f4887 None admin] RBAC: Authorizing `identity:get_user(user_id=6eafabdd3d154b909564ff40e7393e5d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:12.280268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6770509-8424-4b82-803e-462c5d9f4887 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:12.281916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 590/1190] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:12 2022] GET /identity/v3/users/6eafabdd3d154b909564ff40e7393e5d => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:12.287923 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dedb1-47e2-4436-bfe7-5673dcbc34cc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:12.389622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dedb1-47e2-4436-bfe7-5673dcbc34cc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:12.423190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a92432f4-f9ce-44d4-8a43-633956c0d0e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:12.424041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92432f4-f9ce-44d4-8a43-633956c0d0e2 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:12.424327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92432f4-f9ce-44d4-8a43-633956c0d0e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:12.424606 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92432f4-f9ce-44d4-8a43-633956c0d0e2 None admin] PATH_INFO: `/v3/projects/d70c95a3987f429e86092940352f8c6e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:12.426520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92432f4-f9ce-44d4-8a43-633956c0d0e2 None admin] RBAC: Authorizing `identity:get_project(project_id=d70c95a3987f429e86092940352f8c6e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:12.427290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a92432f4-f9ce-44d4-8a43-633956c0d0e2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:12.429110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 601/1191] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:12 2022] GET /identity/v3/projects/d70c95a3987f429e86092940352f8c6e => generated 361 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:12.435138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6770509-8424-4b82-803e-462c5d9f4887 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:12.541032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6770509-8424-4b82-803e-462c5d9f4887 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:12.575534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786d6247-9d91-4065-9e5f-30054f9f3076 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:12.576426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6247-9d91-4065-9e5f-30054f9f3076 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:12.576709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6247-9d91-4065-9e5f-30054f9f3076 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:12.576984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786d6247-9d91-4065-9e5f-30054f9f3076 None admin] PATH_INFO: `/v3/projects/d70c95a3987f429e86092940352f8c6e/users/6eafabdd3d154b909564ff40e7393e5d/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:12.579125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d6247-9d91-4065-9e5f-30054f9f3076 None admin] RBAC: Authorizing `identity:create_grant(project_id=d70c95a3987f429e86092940352f8c6e, user_id=6eafabdd3d154b909564ff40e7393e5d, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:12.580557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786d6247-9d91-4065-9e5f-30054f9f3076 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:12.594735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 591/1192] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:12:12 2022] PUT /identity/v3/projects/d70c95a3987f429e86092940352f8c6e/users/6eafabdd3d154b909564ff40e7393e5d/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:12:14.620172 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e60705-1615-4a23-bf44-d9e4c7ddbcd6 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:14.620695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e60705-1615-4a23-bf44-d9e4c7ddbcd6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:14.620695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e60705-1615-4a23-bf44-d9e4c7ddbcd6 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:14.622024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 602/1193] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:12:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:12:14.627447 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f82c8-53ac-4307-8a57-74ec4105906b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:14.627871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f82c8-53ac-4307-8a57-74ec4105906b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:14.627990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6f82c8-53ac-4307-8a57-74ec4105906b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:14.645323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7c6f82c8-53ac-4307-8a57-74ec4105906b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:12:14.646552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7c6f82c8-53ac-4307-8a57-74ec4105906b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:12:14.654334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:14.715935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 592/1194] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:12:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:12:14.739691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e60705-1615-4a23-bf44-d9e4c7ddbcd6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:14.751861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:14.848592 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e60705-1615-4a23-bf44-d9e4c7ddbcd6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:14.884691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d70896-582c-4773-b290-0fe22ac0b162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:14.885360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d70896-582c-4773-b290-0fe22ac0b162 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:14.885672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d70896-582c-4773-b290-0fe22ac0b162 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:14.885947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d70896-582c-4773-b290-0fe22ac0b162 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:14.896158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d70896-582c-4773-b290-0fe22ac0b162 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f7d70896-582c-4773-b290-0fe22ac0b162 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:12:14.903787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:12:14.904965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:12:14.906068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 603/1195] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:12:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:12:14.911111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f82c8-53ac-4307-8a57-74ec4105906b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:15.013953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6f82c8-53ac-4307-8a57-74ec4105906b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:15.048966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f504fed-4b3c-431c-be59-894ff097cccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:15.049662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f504fed-4b3c-431c-be59-894ff097cccd None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:15.049949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f504fed-4b3c-431c-be59-894ff097cccd None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:15.050222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f504fed-4b3c-431c-be59-894ff097cccd None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:15.050964 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f504fed-4b3c-431c-be59-894ff097cccd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:15.051270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f504fed-4b3c-431c-be59-894ff097cccd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:15.052645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f504fed-4b3c-431c-be59-894ff097cccd None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:15.061619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 593/1196] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:12:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:15.067992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d70896-582c-4773-b290-0fe22ac0b162 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:15.141102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:15.176788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d70896-582c-4773-b290-0fe22ac0b162 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:15.217088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4755be4-b85b-4cfc-9ff8-2b165c68f602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:15.217976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4755be4-b85b-4cfc-9ff8-2b165c68f602 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:15.218258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4755be4-b85b-4cfc-9ff8-2b165c68f602 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:15.218527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4755be4-b85b-4cfc-9ff8-2b165c68f602 None admin] PATH_INFO: `/v3/users/6eafabdd3d154b909564ff40e7393e5d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:15.219702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4755be4-b85b-4cfc-9ff8-2b165c68f602 None admin] RBAC: Authorizing `identity:get_user(user_id=6eafabdd3d154b909564ff40e7393e5d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:15.221140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4755be4-b85b-4cfc-9ff8-2b165c68f602 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:15.223167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 604/1197] 10.222.0.69 () {56 vars in 1177 bytes} [Wed Jul 20 19:12:15 2022] GET /identity/v3/users/6eafabdd3d154b909564ff40e7393e5d => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:15.229874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f504fed-4b3c-431c-be59-894ff097cccd None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:15.233783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:15.337584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f504fed-4b3c-431c-be59-894ff097cccd None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:15.374301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ad4d8-6d37-4a7e-a237-38bf220d9396 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:15.375046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ad4d8-6d37-4a7e-a237-38bf220d9396 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:15.375350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ad4d8-6d37-4a7e-a237-38bf220d9396 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:15.375712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425ad4d8-6d37-4a7e-a237-38bf220d9396 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:15.450898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ad4d8-6d37-4a7e-a237-38bf220d9396 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:15.452553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-425ad4d8-6d37-4a7e-a237-38bf220d9396 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:15.455296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:15.627848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 594/1198] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:12:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:12:15.634567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4755be4-b85b-4cfc-9ff8-2b165c68f602 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:15.754687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4755be4-b85b-4cfc-9ff8-2b165c68f602 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:15.794428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bde9429-10b2-40a1-8457-b3ea60d93e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:15.795356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde9429-10b2-40a1-8457-b3ea60d93e01 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:15.795681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde9429-10b2-40a1-8457-b3ea60d93e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:15.796003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bde9429-10b2-40a1-8457-b3ea60d93e01 None admin] PATH_INFO: `/v3/projects/d70c95a3987f429e86092940352f8c6e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:15.797314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde9429-10b2-40a1-8457-b3ea60d93e01 None admin] RBAC: Authorizing `identity:get_project(project_id=d70c95a3987f429e86092940352f8c6e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:15.798862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bde9429-10b2-40a1-8457-b3ea60d93e01 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:15.800705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 605/1199] 10.222.0.69 () {56 vars in 1186 bytes} [Wed Jul 20 19:12:15 2022] GET /identity/v3/projects/d70c95a3987f429e86092940352f8c6e => generated 361 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:15.808754 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ad4d8-6d37-4a7e-a237-38bf220d9396 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:12:15.941340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-425ad4d8-6d37-4a7e-a237-38bf220d9396 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:12:15.944455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:15.982564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09dc5fd-31b9-4f4a-9b0d-dc0fc9f53ada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:12:15.983252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dc5fd-31b9-4f4a-9b0d-dc0fc9f53ada None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:12:15.983526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dc5fd-31b9-4f4a-9b0d-dc0fc9f53ada None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:12:15.983805 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09dc5fd-31b9-4f4a-9b0d-dc0fc9f53ada None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:12:15.984629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09dc5fd-31b9-4f4a-9b0d-dc0fc9f53ada None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=d70c95a3987f429e86092940352f8c6e, user.id=6eafabdd3d154b909564ff40e7393e5d) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:12:15.984936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09dc5fd-31b9-4f4a-9b0d-dc0fc9f53ada None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:12:15.986407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09dc5fd-31b9-4f4a-9b0d-dc0fc9f53ada None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:12:15.994026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 595/1200] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:12:15 2022] GET /identity/v3/role_assignments?user.id=6eafabdd3d154b909564ff40e7393e5d&scope.project.id=d70c95a3987f429e86092940352f8c6e&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:12:25.789787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:12:25.989957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=16 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:11.740379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca3a3c-28c2-440f-96f5-b125b3100d1b None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:11.740985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca3a3c-28c2-440f-96f5-b125b3100d1b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:11.740985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca3a3c-28c2-440f-96f5-b125b3100d1b None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:11.742746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 606/1201] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:11.748569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547de87c-68de-4f92-9555-e85333da3e2c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:11.749138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547de87c-68de-4f92-9555-e85333da3e2c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:11.749138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547de87c-68de-4f92-9555-e85333da3e2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:11.771279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-547de87c-68de-4f92-9555-e85333da3e2c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:11.772634 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-547de87c-68de-4f92-9555-e85333da3e2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:11.868093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 596/1202] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:11.898848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca3a3c-28c2-440f-96f5-b125b3100d1b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:12.043329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca3a3c-28c2-440f-96f5-b125b3100d1b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:12.084298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3e30e-24a2-41aa-97c8-3dd5f5e458a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:12.085318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3e30e-24a2-41aa-97c8-3dd5f5e458a4 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:12.085770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3e30e-24a2-41aa-97c8-3dd5f5e458a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:12.086166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae3e30e-24a2-41aa-97c8-3dd5f5e458a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:12.099199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3e30e-24a2-41aa-97c8-3dd5f5e458a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:12.100693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae3e30e-24a2-41aa-97c8-3dd5f5e458a4 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-dae3e30e-24a2-41aa-97c8-3dd5f5e458a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:12.121028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:12.122367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:12.123351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:12.123351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:12.123351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:12.123351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:12.129934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 607/1203] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:12.130260 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547de87c-68de-4f92-9555-e85333da3e2c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:12.284448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547de87c-68de-4f92-9555-e85333da3e2c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:12.338173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:12.347004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6586c6c1-79dd-4e31-a76d-d5807541ac32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:12.348252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586c6c1-79dd-4e31-a76d-d5807541ac32 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:12.348689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586c6c1-79dd-4e31-a76d-d5807541ac32 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:12.348996 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6586c6c1-79dd-4e31-a76d-d5807541ac32 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:12.349807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586c6c1-79dd-4e31-a76d-d5807541ac32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:12.350350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586c6c1-79dd-4e31-a76d-d5807541ac32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:12.351816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6586c6c1-79dd-4e31-a76d-d5807541ac32 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:12.366511 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 597/1204] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:12.373040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3e30e-24a2-41aa-97c8-3dd5f5e458a4 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:12.448682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:12.520616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae3e30e-24a2-41aa-97c8-3dd5f5e458a4 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:12.567078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110918e8-a16a-4328-b8cd-f3f0eb1b27a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:12.567887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110918e8-a16a-4328-b8cd-f3f0eb1b27a8 None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:12.568093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110918e8-a16a-4328-b8cd-f3f0eb1b27a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:12.568365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110918e8-a16a-4328-b8cd-f3f0eb1b27a8 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:12.569285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110918e8-a16a-4328-b8cd-f3f0eb1b27a8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:12.571495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110918e8-a16a-4328-b8cd-f3f0eb1b27a8 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:12.595105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 608/1205] 10.222.0.69 () {60 vars in 1132 bytes} [Wed Jul 20 19:13:12 2022] POST /identity/v3/users => generated 251 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:13:14.947024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa55aec-2fca-4a4b-9fd3-c7f137bd51d0 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:14.947578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa55aec-2fca-4a4b-9fd3-c7f137bd51d0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:14.947578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa55aec-2fca-4a4b-9fd3-c7f137bd51d0 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:14.948978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 598/1206] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:14.956090 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873aa90-70b0-4e70-b9e5-eb0839bff123 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:14.956624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873aa90-70b0-4e70-b9e5-eb0839bff123 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:14.956726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8873aa90-70b0-4e70-b9e5-eb0839bff123 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:14.979288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8873aa90-70b0-4e70-b9e5-eb0839bff123 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:14.980516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8873aa90-70b0-4e70-b9e5-eb0839bff123 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:15.074702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 609/1207] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:15.100440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa55aec-2fca-4a4b-9fd3-c7f137bd51d0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:15.235796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa55aec-2fca-4a4b-9fd3-c7f137bd51d0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:15.283919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d412a9-832d-44d5-bd82-2307bb8c96e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:15.284894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d412a9-832d-44d5-bd82-2307bb8c96e9 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:15.285207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d412a9-832d-44d5-bd82-2307bb8c96e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:15.285641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d412a9-832d-44d5-bd82-2307bb8c96e9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:15.300611 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d412a9-832d-44d5-bd82-2307bb8c96e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-17d412a9-832d-44d5-bd82-2307bb8c96e9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:15.311628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:15.312855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:15.313958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 599/1208] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:13:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:15.321353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873aa90-70b0-4e70-b9e5-eb0839bff123 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:15.376572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:15.465848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8873aa90-70b0-4e70-b9e5-eb0839bff123 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:15.518849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae91f7-0fed-4da9-bc4b-eb3dbee7c135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:15.520206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae91f7-0fed-4da9-bc4b-eb3dbee7c135 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:15.520588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae91f7-0fed-4da9-bc4b-eb3dbee7c135 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:15.520964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ae91f7-0fed-4da9-bc4b-eb3dbee7c135 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:15.522130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae91f7-0fed-4da9-bc4b-eb3dbee7c135 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:15.522492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae91f7-0fed-4da9-bc4b-eb3dbee7c135 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:15.523901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ae91f7-0fed-4da9-bc4b-eb3dbee7c135 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:15.534997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 610/1209] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:13:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:15.541948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d412a9-832d-44d5-bd82-2307bb8c96e9 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:15.613561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:15.690164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d412a9-832d-44d5-bd82-2307bb8c96e9 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:15.747918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aab8f-dcdc-47d6-b571-ff357d9dc940 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:15.748982 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aab8f-dcdc-47d6-b571-ff357d9dc940 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:15.749446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aab8f-dcdc-47d6-b571-ff357d9dc940 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:15.749707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934aab8f-dcdc-47d6-b571-ff357d9dc940 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:15.771236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aab8f-dcdc-47d6-b571-ff357d9dc940 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:15.773209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934aab8f-dcdc-47d6-b571-ff357d9dc940 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-934aab8f-dcdc-47d6-b571-ff357d9dc940 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:15.794796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:15.795979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:15.797120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:15.797120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:15.797120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:15.797120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:15.798838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 600/1210] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:15.806581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae91f7-0fed-4da9-bc4b-eb3dbee7c135 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:15.933498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:15.937307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ae91f7-0fed-4da9-bc4b-eb3dbee7c135 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:15.986555 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8c59e-1350-4d77-b22d-da111cb79b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:15.987386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8c59e-1350-4d77-b22d-da111cb79b18 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:15.987610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8c59e-1350-4d77-b22d-da111cb79b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:15.988590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf8c59e-1350-4d77-b22d-da111cb79b18 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:15.989684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8c59e-1350-4d77-b22d-da111cb79b18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:15.990121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8c59e-1350-4d77-b22d-da111cb79b18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:15.992602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf8c59e-1350-4d77-b22d-da111cb79b18 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:16.015331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 611/1211] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:16.029623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aab8f-dcdc-47d6-b571-ff357d9dc940 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:16.177698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934aab8f-dcdc-47d6-b571-ff357d9dc940 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:16.180442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:16.231970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97405e-5a2d-4296-b4ad-bdc2ce23b938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:16.232786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97405e-5a2d-4296-b4ad-bdc2ce23b938 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:16.233155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97405e-5a2d-4296-b4ad-bdc2ce23b938 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:16.233479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b97405e-5a2d-4296-b4ad-bdc2ce23b938 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:16.244211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97405e-5a2d-4296-b4ad-bdc2ce23b938 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:16.246259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b97405e-5a2d-4296-b4ad-bdc2ce23b938 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1b97405e-5a2d-4296-b4ad-bdc2ce23b938 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:13:16.266467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:16.267774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:16.268900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 601/1212] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:16.277296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8c59e-1350-4d77-b22d-da111cb79b18 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:16.422429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf8c59e-1350-4d77-b22d-da111cb79b18 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:16.465635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506d088-ef6c-4577-85eb-b485d937b915 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:16.466335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506d088-ef6c-4577-85eb-b485d937b915 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:16.466627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506d088-ef6c-4577-85eb-b485d937b915 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:16.466902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0506d088-ef6c-4577-85eb-b485d937b915 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:16.467804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506d088-ef6c-4577-85eb-b485d937b915 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:16.468106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506d088-ef6c-4577-85eb-b485d937b915 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:16.469713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0506d088-ef6c-4577-85eb-b485d937b915 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:16.496025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 612/1213] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Jul 20 19:13:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:16.502033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97405e-5a2d-4296-b4ad-bdc2ce23b938 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:16.623105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b97405e-5a2d-4296-b4ad-bdc2ce23b938 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:16.678318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45073d8d-d101-4cd6-b2ff-04b720d6474a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:16.679470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45073d8d-d101-4cd6-b2ff-04b720d6474a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:16.679663 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45073d8d-d101-4cd6-b2ff-04b720d6474a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:16.679953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45073d8d-d101-4cd6-b2ff-04b720d6474a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:16.728767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:16.788382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45073d8d-d101-4cd6-b2ff-04b720d6474a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:16.790134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45073d8d-d101-4cd6-b2ff-04b720d6474a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:16.931456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 602/1214] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:13:16.937280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506d088-ef6c-4577-85eb-b485d937b915 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:16.959556 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:17.062279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0506d088-ef6c-4577-85eb-b485d937b915 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:17.106280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a6050-5678-4f8c-a5c8-5e527b54f2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:17.107174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a6050-5678-4f8c-a5c8-5e527b54f2db None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:17.107686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a6050-5678-4f8c-a5c8-5e527b54f2db None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:17.108079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5a6050-5678-4f8c-a5c8-5e527b54f2db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:17.123619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a6050-5678-4f8c-a5c8-5e527b54f2db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:17.125803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5a6050-5678-4f8c-a5c8-5e527b54f2db None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-0e5a6050-5678-4f8c-a5c8-5e527b54f2db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:17.149869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:17.151078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:17.152215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:17.152215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:17.152215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:17.152215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:17.153343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 613/1215] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:17.159635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45073d8d-d101-4cd6-b2ff-04b720d6474a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:17.253764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:17.320709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45073d8d-d101-4cd6-b2ff-04b720d6474a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:17.373748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a46f71-41f9-4c19-9569-e9b433bec931 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:17.374488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a46f71-41f9-4c19-9569-e9b433bec931 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:17.374708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a46f71-41f9-4c19-9569-e9b433bec931 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:17.374983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a46f71-41f9-4c19-9569-e9b433bec931 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:17.375918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a46f71-41f9-4c19-9569-e9b433bec931 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:17.376187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a46f71-41f9-4c19-9569-e9b433bec931 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:17.377764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a46f71-41f9-4c19-9569-e9b433bec931 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:17.392055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 603/1216] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:17.399025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a6050-5678-4f8c-a5c8-5e527b54f2db None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:17.495262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:17.544805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5a6050-5678-4f8c-a5c8-5e527b54f2db None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:17.599029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fbfe0-4540-4a18-b09b-8e4b36fa33ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:17.600064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fbfe0-4540-4a18-b09b-8e4b36fa33ab None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:17.600385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fbfe0-4540-4a18-b09b-8e4b36fa33ab None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:17.600813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816fbfe0-4540-4a18-b09b-8e4b36fa33ab None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:17.610436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fbfe0-4540-4a18-b09b-8e4b36fa33ab None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:17.612362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816fbfe0-4540-4a18-b09b-8e4b36fa33ab None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-816fbfe0-4540-4a18-b09b-8e4b36fa33ab None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:13:17.629670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:17.631525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:17.633441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:13:17.633441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:13:17.633441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:13:17.633441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:13:17.633441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:13:17.633441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:13:17.633441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:17.633441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:17.633441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 614/1217] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:17.637023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a46f71-41f9-4c19-9569-e9b433bec931 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:17.777450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a46f71-41f9-4c19-9569-e9b433bec931 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:17.808081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:17.826951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a221b7-14a2-4462-b667-920fe515e7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:17.827778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a221b7-14a2-4462-b667-920fe515e7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:17.828163 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a221b7-14a2-4462-b667-920fe515e7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:17.828586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a221b7-14a2-4462-b667-920fe515e7d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:17.829386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a221b7-14a2-4462-b667-920fe515e7d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:17.829864 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a221b7-14a2-4462-b667-920fe515e7d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:17.831735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a221b7-14a2-4462-b667-920fe515e7d8 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:17.845003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 604/1218] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:13:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:17.854246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fbfe0-4540-4a18-b09b-8e4b36fa33ab None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:18.005010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816fbfe0-4540-4a18-b09b-8e4b36fa33ab None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:18.042391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:18.050963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cea4cd-26df-4a10-8a25-2f03cc6f4ced None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:18.051591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cea4cd-26df-4a10-8a25-2f03cc6f4ced None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:18.051882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cea4cd-26df-4a10-8a25-2f03cc6f4ced None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:18.052158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cea4cd-26df-4a10-8a25-2f03cc6f4ced None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:18.052990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cea4cd-26df-4a10-8a25-2f03cc6f4ced None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=d4d7142663cb4d1f92ecc4da831ac6b6) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:18.053305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cea4cd-26df-4a10-8a25-2f03cc6f4ced None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:18.055127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cea4cd-26df-4a10-8a25-2f03cc6f4ced None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:18.062050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 615/1219] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:17 2022] GET /identity/v3/role_assignments?user.id=d4d7142663cb4d1f92ecc4da831ac6b6&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 259 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:20.354196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d62f82-8dc8-4fbd-8615-cf6ae6db0407 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:20.354762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d62f82-8dc8-4fbd-8615-cf6ae6db0407 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:20.354868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d62f82-8dc8-4fbd-8615-cf6ae6db0407 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:20.356191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 605/1220] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:20.361926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7104f-c563-41d2-9ac0-58d9cc387599 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:20.362392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7104f-c563-41d2-9ac0-58d9cc387599 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:20.362511 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d7104f-c563-41d2-9ac0-58d9cc387599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:20.381142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-34d7104f-c563-41d2-9ac0-58d9cc387599 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:20.382419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-34d7104f-c563-41d2-9ac0-58d9cc387599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:20.460052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 616/1221] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:20.485745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d62f82-8dc8-4fbd-8615-cf6ae6db0407 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:20.601309 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d62f82-8dc8-4fbd-8615-cf6ae6db0407 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:20.636945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a29b3-0b1a-4f60-9332-b884a07cd042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:20.637776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a29b3-0b1a-4f60-9332-b884a07cd042 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:20.638053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a29b3-0b1a-4f60-9332-b884a07cd042 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:20.638324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8a29b3-0b1a-4f60-9332-b884a07cd042 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:20.648553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8a29b3-0b1a-4f60-9332-b884a07cd042 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-3f8a29b3-0b1a-4f60-9332-b884a07cd042 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:20.656267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:20.657524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:20.658444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 606/1222] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:13:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:20.664957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7104f-c563-41d2-9ac0-58d9cc387599 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:20.787139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d7104f-c563-41d2-9ac0-58d9cc387599 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:20.812183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:20.827138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c2dfa-d661-4d8c-aff2-57325a776f86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:20.827809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c2dfa-d661-4d8c-aff2-57325a776f86 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:20.828093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c2dfa-d661-4d8c-aff2-57325a776f86 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:20.828372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755c2dfa-d661-4d8c-aff2-57325a776f86 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:20.829133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c2dfa-d661-4d8c-aff2-57325a776f86 None admin] RBAC: Adding query filter params (name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:20.829442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c2dfa-d661-4d8c-aff2-57325a776f86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:20.830844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755c2dfa-d661-4d8c-aff2-57325a776f86 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:20.839492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 617/1223] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:13:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:20.848248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a29b3-0b1a-4f60-9332-b884a07cd042 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:20.898093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:20.975258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8a29b3-0b1a-4f60-9332-b884a07cd042 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:21.015646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d993010-2fd9-4a16-9f5a-b949d3df8a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:21.016482 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d993010-2fd9-4a16-9f5a-b949d3df8a36 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:21.016766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d993010-2fd9-4a16-9f5a-b949d3df8a36 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:21.017044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d993010-2fd9-4a16-9f5a-b949d3df8a36 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:21.029229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d993010-2fd9-4a16-9f5a-b949d3df8a36 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:21.030779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d993010-2fd9-4a16-9f5a-b949d3df8a36 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-2d993010-2fd9-4a16-9f5a-b949d3df8a36 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:21.049316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:21.050442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:21.051535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:21.051535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:21.051535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:21.051535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:21.051535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 607/1224] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:21.057543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c2dfa-d661-4d8c-aff2-57325a776f86 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:21.188359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755c2dfa-d661-4d8c-aff2-57325a776f86 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:21.226317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386a65c-64a6-4f91-8c8a-ee105692c2b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:21.226984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386a65c-64a6-4f91-8c8a-ee105692c2b5 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:21.227264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386a65c-64a6-4f91-8c8a-ee105692c2b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:21.227565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1386a65c-64a6-4f91-8c8a-ee105692c2b5 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:21.228281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386a65c-64a6-4f91-8c8a-ee105692c2b5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:21.228581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386a65c-64a6-4f91-8c8a-ee105692c2b5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:21.230005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1386a65c-64a6-4f91-8c8a-ee105692c2b5 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:21.242539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 618/1225] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:21.248330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d993010-2fd9-4a16-9f5a-b949d3df8a36 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:21.361600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:21.370343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d993010-2fd9-4a16-9f5a-b949d3df8a36 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:21.418783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff47229-dade-48ae-9874-9f844099c19f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:21.419101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff47229-dade-48ae-9874-9f844099c19f None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:21.419425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff47229-dade-48ae-9874-9f844099c19f None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:21.419820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff47229-dade-48ae-9874-9f844099c19f None admin] PATH_INFO: `/v3/users/placement` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:21.429281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff47229-dade-48ae-9874-9f844099c19f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:21.430804 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff47229-dade-48ae-9874-9f844099c19f None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-6ff47229-dade-48ae-9874-9f844099c19f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:13:21.446780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:21.448732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:21.450118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 608/1226] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:21.454088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386a65c-64a6-4f91-8c8a-ee105692c2b5 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:21.516539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:21.583504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1386a65c-64a6-4f91-8c8a-ee105692c2b5 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:21.630563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de28b5-76db-4565-8d29-08dc695f8e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:21.631184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de28b5-76db-4565-8d29-08dc695f8e89 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:21.631465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de28b5-76db-4565-8d29-08dc695f8e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:21.631737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de28b5-76db-4565-8d29-08dc695f8e89 None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:21.632584 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de28b5-76db-4565-8d29-08dc695f8e89 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:21.632888 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de28b5-76db-4565-8d29-08dc695f8e89 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:21.634329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de28b5-76db-4565-8d29-08dc695f8e89 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:21.655133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 619/1227] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Jul 20 19:13:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:21.662767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff47229-dade-48ae-9874-9f844099c19f None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:21.788045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff47229-dade-48ae-9874-9f844099c19f None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:21.824534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e2c3b-fe55-48ab-8205-1f51805b6ab2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:21.825345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2c3b-fe55-48ab-8205-1f51805b6ab2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:21.825671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2c3b-fe55-48ab-8205-1f51805b6ab2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:21.825951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31e2c3b-fe55-48ab-8205-1f51805b6ab2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:21.837860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e2c3b-fe55-48ab-8205-1f51805b6ab2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:21.839591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31e2c3b-fe55-48ab-8205-1f51805b6ab2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-f31e2c3b-fe55-48ab-8205-1f51805b6ab2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:21.866059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:21.867110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:21.868036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:21.868036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:21.868036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:21.868036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:21.868036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 609/1228] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:21.868323 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de28b5-76db-4565-8d29-08dc695f8e89 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:21.984092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de28b5-76db-4565-8d29-08dc695f8e89 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:21.986760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:22.029830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5b44e-e311-4494-bfae-02398f4f6acd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:22.030607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b44e-e311-4494-bfae-02398f4f6acd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:22.030960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b44e-e311-4494-bfae-02398f4f6acd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:22.031308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d5b44e-e311-4494-bfae-02398f4f6acd None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:22.032985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5b44e-e311-4494-bfae-02398f4f6acd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:22.033305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5b44e-e311-4494-bfae-02398f4f6acd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:22.035595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d5b44e-e311-4494-bfae-02398f4f6acd None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:22.053751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 620/1229] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:22.054895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e2c3b-fe55-48ab-8205-1f51805b6ab2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:22.064768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:22.180495 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31e2c3b-fe55-48ab-8205-1f51805b6ab2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:22.229896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6c720-5e5f-4e61-b6a7-3201573edfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:22.230971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6c720-5e5f-4e61-b6a7-3201573edfee None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:22.231346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6c720-5e5f-4e61-b6a7-3201573edfee None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:22.231616 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6c720-5e5f-4e61-b6a7-3201573edfee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:22.240337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6c720-5e5f-4e61-b6a7-3201573edfee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:22.241834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6c720-5e5f-4e61-b6a7-3201573edfee None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-13a6c720-5e5f-4e61-b6a7-3201573edfee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:13:22.256233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:22.257383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:22.258497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:13:22.258497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:13:22.258497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:13:22.258497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:13:22.258497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:13:22.258497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:13:22.258497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:22.258497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:22.258497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 610/1230] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:22.263250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5b44e-e311-4494-bfae-02398f4f6acd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:22.384980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d5b44e-e311-4494-bfae-02398f4f6acd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:22.424500 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a03aaa1-dbab-4361-80ce-8842b9504ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:22.425868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03aaa1-dbab-4361-80ce-8842b9504ff7 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:22.426164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03aaa1-dbab-4361-80ce-8842b9504ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:22.426435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a03aaa1-dbab-4361-80ce-8842b9504ff7 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:22.427227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03aaa1-dbab-4361-80ce-8842b9504ff7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:22.427544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03aaa1-dbab-4361-80ce-8842b9504ff7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:22.429466 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a03aaa1-dbab-4361-80ce-8842b9504ff7 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:22.447706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 621/1231] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:13:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:22.453816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6c720-5e5f-4e61-b6a7-3201573edfee None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:22.540981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:22.590789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6c720-5e5f-4e61-b6a7-3201573edfee None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:22.633068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad02b19b-5ac7-48b5-9322-0cc5487f3934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:22.634100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b19b-5ac7-48b5-9322-0cc5487f3934 None admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:22.634437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b19b-5ac7-48b5-9322-0cc5487f3934 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:22.634762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad02b19b-5ac7-48b5-9322-0cc5487f3934 None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/d4d7142663cb4d1f92ecc4da831ac6b6/roles/5b019d81d9cf4a16b6c6d1caafec9081` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:22.651872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b19b-5ac7-48b5-9322-0cc5487f3934 None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=d4d7142663cb4d1f92ecc4da831ac6b6, role_id=5b019d81d9cf4a16b6c6d1caafec9081)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:22.653327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad02b19b-5ac7-48b5-9322-0cc5487f3934 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:22.667873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 611/1232] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:13:22 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/d4d7142663cb4d1f92ecc4da831ac6b6/roles/5b019d81d9cf4a16b6c6d1caafec9081 => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:13:24.961761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7124522-a4da-47ed-9dec-2222541fe985 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:24.962335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7124522-a4da-47ed-9dec-2222541fe985 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:24.962450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7124522-a4da-47ed-9dec-2222541fe985 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:24.963873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 622/1233] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:24.970524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ba4f2-6c36-4c0e-b65f-a294fe4512ac None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:24.971023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ba4f2-6c36-4c0e-b65f-a294fe4512ac None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:24.971118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0ba4f2-6c36-4c0e-b65f-a294fe4512ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:24.995141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ba0ba4f2-6c36-4c0e-b65f-a294fe4512ac None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:24.996284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ba0ba4f2-6c36-4c0e-b65f-a294fe4512ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:25.100774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 612/1234] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:25.124822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7124522-a4da-47ed-9dec-2222541fe985 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:25.156578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:25.252324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7124522-a4da-47ed-9dec-2222541fe985 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:25.301866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe0ad7-14db-41f6-be18-c1d6d150a4dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:25.302881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0ad7-14db-41f6-be18-c1d6d150a4dc None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:25.303118 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0ad7-14db-41f6-be18-c1d6d150a4dc None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:25.303534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fe0ad7-14db-41f6-be18-c1d6d150a4dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:25.324825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5fe0ad7-14db-41f6-be18-c1d6d150a4dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-a5fe0ad7-14db-41f6-be18-c1d6d150a4dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:25.333443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:25.334583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:25.335490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 623/1235] 10.222.0.69 () {56 vars in 1102 bytes} [Wed Jul 20 19:13:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:25.340933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ba4f2-6c36-4c0e-b65f-a294fe4512ac None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:25.399996 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:25.466473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0ba4f2-6c36-4c0e-b65f-a294fe4512ac None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:25.506149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa86352-5b8c-433a-ba11-38a5fc1736ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:25.506744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86352-5b8c-433a-ba11-38a5fc1736ea None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:25.507021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86352-5b8c-433a-ba11-38a5fc1736ea None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:25.507295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa86352-5b8c-433a-ba11-38a5fc1736ea None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:25.508036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa86352-5b8c-433a-ba11-38a5fc1736ea None admin] RBAC: Adding query filter params (name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:25.508359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa86352-5b8c-433a-ba11-38a5fc1736ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:25.509758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fa86352-5b8c-433a-ba11-38a5fc1736ea None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:25.519658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 613/1236] 10.222.0.69 () {56 vars in 1103 bytes} [Wed Jul 20 19:13:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:25.525429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe0ad7-14db-41f6-be18-c1d6d150a4dc None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:25.620111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:25.650316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fe0ad7-14db-41f6-be18-c1d6d150a4dc None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:25.702181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c099ec-6106-4d3b-89d5-017a22da3a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:25.703383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c099ec-6106-4d3b-89d5-017a22da3a29 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:25.703771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c099ec-6106-4d3b-89d5-017a22da3a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:25.704132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c099ec-6106-4d3b-89d5-017a22da3a29 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:25.720016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c099ec-6106-4d3b-89d5-017a22da3a29 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:25.722229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c099ec-6106-4d3b-89d5-017a22da3a29 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-c8c099ec-6106-4d3b-89d5-017a22da3a29 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:25.744525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:25.745903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:25.747181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:25.747181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:25.747181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:25.747181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:25.752427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa86352-5b8c-433a-ba11-38a5fc1736ea None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:25.752597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 624/1237] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:25.871544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fa86352-5b8c-433a-ba11-38a5fc1736ea None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:25.884739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:25.915284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965fcb2-be2f-43da-8854-ee47f6faaa4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:25.915952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965fcb2-be2f-43da-8854-ee47f6faaa4a None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:25.916262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965fcb2-be2f-43da-8854-ee47f6faaa4a None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:25.916522 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f965fcb2-be2f-43da-8854-ee47f6faaa4a None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:25.917258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965fcb2-be2f-43da-8854-ee47f6faaa4a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:25.917601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965fcb2-be2f-43da-8854-ee47f6faaa4a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:25.919249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f965fcb2-be2f-43da-8854-ee47f6faaa4a None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:25.934774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 614/1238] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:25.941951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c099ec-6106-4d3b-89d5-017a22da3a29 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:26.067869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c099ec-6106-4d3b-89d5-017a22da3a29 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:26.121885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7868-727c-468e-95ed-4c701eb46e26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:26.122683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7868-727c-468e-95ed-4c701eb46e26 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:26.123002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7868-727c-468e-95ed-4c701eb46e26 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:26.123260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89bf7868-727c-468e-95ed-4c701eb46e26 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:26.126450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:26.132412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf7868-727c-468e-95ed-4c701eb46e26 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:26.134461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89bf7868-727c-468e-95ed-4c701eb46e26 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-89bf7868-727c-468e-95ed-4c701eb46e26 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:13:26.150797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:26.152059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:26.153397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 625/1239] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:26.158028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965fcb2-be2f-43da-8854-ee47f6faaa4a None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:26.285064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f965fcb2-be2f-43da-8854-ee47f6faaa4a None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:26.332596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b8688-30fe-4f88-85aa-d876ee0dc6fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:26.333457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b8688-30fe-4f88-85aa-d876ee0dc6fb None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:26.333779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b8688-30fe-4f88-85aa-d876ee0dc6fb None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:26.334062 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8b8688-30fe-4f88-85aa-d876ee0dc6fb None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:26.335214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b8688-30fe-4f88-85aa-d876ee0dc6fb None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:26.335532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b8688-30fe-4f88-85aa-d876ee0dc6fb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:26.336993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8b8688-30fe-4f88-85aa-d876ee0dc6fb None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:26.357349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 615/1240] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Jul 20 19:13:26 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:26.364171 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7868-727c-468e-95ed-4c701eb46e26 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:26.494058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89bf7868-727c-468e-95ed-4c701eb46e26 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:26.543668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d279f2-e247-4610-9303-4f0a165fca47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:26.544322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d279f2-e247-4610-9303-4f0a165fca47 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:26.544629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d279f2-e247-4610-9303-4f0a165fca47 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:26.544906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d279f2-e247-4610-9303-4f0a165fca47 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:26.612135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:26.619898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d279f2-e247-4610-9303-4f0a165fca47 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:26.621362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d279f2-e247-4610-9303-4f0a165fca47 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:26.740583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 626/1241] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:26 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:13:26.747103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b8688-30fe-4f88-85aa-d876ee0dc6fb None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:26.776876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:26.875020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8b8688-30fe-4f88-85aa-d876ee0dc6fb None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:26.917654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f55f7f-f9e0-47c5-895a-adadbc55bd73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:26.918537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f55f7f-f9e0-47c5-895a-adadbc55bd73 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:26.918753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f55f7f-f9e0-47c5-895a-adadbc55bd73 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:26.919031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f55f7f-f9e0-47c5-895a-adadbc55bd73 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:26.930243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f55f7f-f9e0-47c5-895a-adadbc55bd73 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:26.931787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f55f7f-f9e0-47c5-895a-adadbc55bd73 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-64f55f7f-f9e0-47c5-895a-adadbc55bd73 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:26.952801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:26.954066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:26.955128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:26.955128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:26.955128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:26.955128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:26.955128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 616/1242] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:26.962004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d279f2-e247-4610-9303-4f0a165fca47 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:27.094214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:27.102723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d279f2-e247-4610-9303-4f0a165fca47 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:27.140947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b903176-9737-46db-9663-7a61aaa71337 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:27.141676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b903176-9737-46db-9663-7a61aaa71337 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:27.141942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b903176-9737-46db-9663-7a61aaa71337 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:27.142305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b903176-9737-46db-9663-7a61aaa71337 None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:27.143025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b903176-9737-46db-9663-7a61aaa71337 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:27.143378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b903176-9737-46db-9663-7a61aaa71337 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:27.145589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b903176-9737-46db-9663-7a61aaa71337 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:27.158745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 627/1243] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:27.165230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f55f7f-f9e0-47c5-895a-adadbc55bd73 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:27.265945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:27.294591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f55f7f-f9e0-47c5-895a-adadbc55bd73 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:27.333943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c884ae-436b-4eee-8118-3324cd0e460b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:27.334757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c884ae-436b-4eee-8118-3324cd0e460b None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:27.335045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c884ae-436b-4eee-8118-3324cd0e460b None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:27.335322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c884ae-436b-4eee-8118-3324cd0e460b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:27.343041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c884ae-436b-4eee-8118-3324cd0e460b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:27.345052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c884ae-436b-4eee-8118-3324cd0e460b None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-41c884ae-436b-4eee-8118-3324cd0e460b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:13:27.359545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:27.360673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:27.361707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:13:27.361707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:13:27.361707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:13:27.361707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:13:27.361707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:13:27.361707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:13:27.361707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:27.361707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:27.361707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 617/1244] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:27.366742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b903176-9737-46db-9663-7a61aaa71337 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:27.489801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b903176-9737-46db-9663-7a61aaa71337 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:27.529146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b20b31ce-5a99-4cad-b677-b04a604e31da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:27.529890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b31ce-5a99-4cad-b677-b04a604e31da None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:27.530175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b31ce-5a99-4cad-b677-b04a604e31da None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:27.530451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b20b31ce-5a99-4cad-b677-b04a604e31da None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:27.531322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b31ce-5a99-4cad-b677-b04a604e31da None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:27.531630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b31ce-5a99-4cad-b677-b04a604e31da None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:27.533216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b20b31ce-5a99-4cad-b677-b04a604e31da None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:27.548089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 628/1245] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:13:27 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:27.554749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c884ae-436b-4eee-8118-3324cd0e460b None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:27.675474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c884ae-436b-4eee-8118-3324cd0e460b None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:27.716359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24001bd4-2fa9-4dec-8bff-5808ed32d020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:27.717037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24001bd4-2fa9-4dec-8bff-5808ed32d020 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:27.717357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24001bd4-2fa9-4dec-8bff-5808ed32d020 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:27.717683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24001bd4-2fa9-4dec-8bff-5808ed32d020 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:27.718530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24001bd4-2fa9-4dec-8bff-5808ed32d020 None admin] RBAC: Adding query filter params (role.id=5b019d81d9cf4a16b6c6d1caafec9081, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=d4d7142663cb4d1f92ecc4da831ac6b6) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:27.718834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24001bd4-2fa9-4dec-8bff-5808ed32d020 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:27.720284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24001bd4-2fa9-4dec-8bff-5808ed32d020 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:27.726827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 618/1246] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:27 2022] GET /identity/v3/role_assignments?user.id=d4d7142663cb4d1f92ecc4da831ac6b6&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=5b019d81d9cf4a16b6c6d1caafec9081 => generated 608 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:30.185777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ad345-afec-40b1-ab9e-bc3716722cb0 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:30.186312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ad345-afec-40b1-ab9e-bc3716722cb0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:30.186450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ad345-afec-40b1-ab9e-bc3716722cb0 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:30.187883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 629/1247] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:30.194818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bed02-4554-47cc-b22b-32016391d1f4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:30.195349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bed02-4554-47cc-b22b-32016391d1f4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:30.195349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879bed02-4554-47cc-b22b-32016391d1f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:30.200763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:30.214876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-879bed02-4554-47cc-b22b-32016391d1f4 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:30.216132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-879bed02-4554-47cc-b22b-32016391d1f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:30.298328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 619/1248] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:30.323271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ad345-afec-40b1-ab9e-bc3716722cb0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:30.348034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:30.453579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ad345-afec-40b1-ab9e-bc3716722cb0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:30.496173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcb67e-9dc9-4fbc-932e-6a4e2bd9194c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:30.496979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcb67e-9dc9-4fbc-932e-6a4e2bd9194c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:30.497265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcb67e-9dc9-4fbc-932e-6a4e2bd9194c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:30.497571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32bcb67e-9dc9-4fbc-932e-6a4e2bd9194c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:30.509567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32bcb67e-9dc9-4fbc-932e-6a4e2bd9194c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-32bcb67e-9dc9-4fbc-932e-6a4e2bd9194c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:30.518777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:30.519988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:30.520885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 630/1249] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:30.527477 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bed02-4554-47cc-b22b-32016391d1f4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:30.655115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879bed02-4554-47cc-b22b-32016391d1f4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:30.692073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:30.698333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6b92c-4d11-498f-8378-29768d2fa1ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:30.699665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6b92c-4d11-498f-8378-29768d2fa1ed None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:30.700000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6b92c-4d11-498f-8378-29768d2fa1ed None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:30.700329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58c6b92c-4d11-498f-8378-29768d2fa1ed None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:30.701230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6b92c-4d11-498f-8378-29768d2fa1ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:30.701591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6b92c-4d11-498f-8378-29768d2fa1ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:30.703404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58c6b92c-4d11-498f-8378-29768d2fa1ed None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:30.717621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 620/1250] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:30.724035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcb67e-9dc9-4fbc-932e-6a4e2bd9194c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:30.816343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:30.846198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32bcb67e-9dc9-4fbc-932e-6a4e2bd9194c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:30.889248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60672463-d7a3-4adb-ab0c-d71abed37cdb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:30.890276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60672463-d7a3-4adb-ab0c-d71abed37cdb None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:30.890532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60672463-d7a3-4adb-ab0c-d71abed37cdb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:30.890801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60672463-d7a3-4adb-ab0c-d71abed37cdb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:30.903740 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60672463-d7a3-4adb-ab0c-d71abed37cdb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:30.905375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60672463-d7a3-4adb-ab0c-d71abed37cdb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-60672463-d7a3-4adb-ab0c-d71abed37cdb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:30.927161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:30.933481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:30.934741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:30.934741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:30.934741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:30.934741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:30.934741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 631/1251] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:30.935120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6b92c-4d11-498f-8378-29768d2fa1ed None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:31.069112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58c6b92c-4d11-498f-8378-29768d2fa1ed None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:31.109714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d4a60-c5f5-49c2-9b24-a8eb672dd7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:31.110386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d4a60-c5f5-49c2-9b24-a8eb672dd7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.110672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d4a60-c5f5-49c2-9b24-a8eb672dd7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.110938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214d4a60-c5f5-49c2-9b24-a8eb672dd7b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.111657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d4a60-c5f5-49c2-9b24-a8eb672dd7b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:31.111960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d4a60-c5f5-49c2-9b24-a8eb672dd7b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:31.113371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214d4a60-c5f5-49c2-9b24-a8eb672dd7b2 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:31.130105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 621/1252] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:31.136508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60672463-d7a3-4adb-ab0c-d71abed37cdb None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:31.259176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60672463-d7a3-4adb-ab0c-d71abed37cdb None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:31.296315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:31.306526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c380bf-2920-4ef3-8e1f-45e3475adaf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:31.307431 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c380bf-2920-4ef3-8e1f-45e3475adaf9 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.307716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c380bf-2920-4ef3-8e1f-45e3475adaf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.307996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c380bf-2920-4ef3-8e1f-45e3475adaf9 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.317563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c380bf-2920-4ef3-8e1f-45e3475adaf9 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:31.318910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c380bf-2920-4ef3-8e1f-45e3475adaf9 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-80c380bf-2920-4ef3-8e1f-45e3475adaf9 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:13:31.334946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:31.336200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:31.337195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 632/1253] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:31 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:31.342303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d4a60-c5f5-49c2-9b24-a8eb672dd7b2 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:31.410194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:31.463174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214d4a60-c5f5-49c2-9b24-a8eb672dd7b2 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:31.505282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71f109-a687-4f20-b40f-a5edff71e063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:31.506516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71f109-a687-4f20-b40f-a5edff71e063 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.506950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71f109-a687-4f20-b40f-a5edff71e063 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.507438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b71f109-a687-4f20-b40f-a5edff71e063 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.508814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71f109-a687-4f20-b40f-a5edff71e063 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:31.509502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71f109-a687-4f20-b40f-a5edff71e063 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:31.511826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b71f109-a687-4f20-b40f-a5edff71e063 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:31.536295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 622/1254] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Jul 20 19:13:31 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:31.544486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c380bf-2920-4ef3-8e1f-45e3475adaf9 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:31.688189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c380bf-2920-4ef3-8e1f-45e3475adaf9 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:31.728283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d50ee-e5c2-488b-b4c6-fe2f0feeaba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:31.729052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d50ee-e5c2-488b-b4c6-fe2f0feeaba3 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:31.729335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d50ee-e5c2-488b-b4c6-fe2f0feeaba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:31.729635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61d50ee-e5c2-488b-b4c6-fe2f0feeaba3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:31.798752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:31.814106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d50ee-e5c2-488b-b4c6-fe2f0feeaba3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:31.815668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61d50ee-e5c2-488b-b4c6-fe2f0feeaba3 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:31.928585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 633/1255] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:31 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 20 19:13:31.936524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71f109-a687-4f20-b40f-a5edff71e063 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.064507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b71f109-a687-4f20-b40f-a5edff71e063 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.079921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:32.109871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b250a3-e9a7-408f-a872-ccc235034bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.110669 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b250a3-e9a7-408f-a872-ccc235034bfa None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.110951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b250a3-e9a7-408f-a872-ccc235034bfa None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.111223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b250a3-e9a7-408f-a872-ccc235034bfa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.124727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b250a3-e9a7-408f-a872-ccc235034bfa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.126701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0b250a3-e9a7-408f-a872-ccc235034bfa None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-b0b250a3-e9a7-408f-a872-ccc235034bfa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:32.150770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:32.151867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:32.152773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:32.152773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:32.152773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:32.152773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:32.152773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 623/1256] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:32.161202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d50ee-e5c2-488b-b4c6-fe2f0feeaba3 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.256684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:32.269231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61d50ee-e5c2-488b-b4c6-fe2f0feeaba3 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.305175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa978-1e9b-411e-81f1-349a7199d1db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.305866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa978-1e9b-411e-81f1-349a7199d1db None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.306146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa978-1e9b-411e-81f1-349a7199d1db None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.306420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eefa978-1e9b-411e-81f1-349a7199d1db None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.307137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa978-1e9b-411e-81f1-349a7199d1db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:32.307444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa978-1e9b-411e-81f1-349a7199d1db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.308805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eefa978-1e9b-411e-81f1-349a7199d1db None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.321615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 634/1257] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:13:32.327494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b250a3-e9a7-408f-a872-ccc235034bfa None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.435186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b250a3-e9a7-408f-a872-ccc235034bfa None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.471012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74cb31-4ff5-49a4-a884-fbc3a359e1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.472171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74cb31-4ff5-49a4-a884-fbc3a359e1be None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.472503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74cb31-4ff5-49a4-a884-fbc3a359e1be None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.472720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a74cb31-4ff5-49a4-a884-fbc3a359e1be None admin] PATH_INFO: `/v3/projects/service` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.479838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74cb31-4ff5-49a4-a884-fbc3a359e1be None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.481482 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a74cb31-4ff5-49a4-a884-fbc3a359e1be None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-1a74cb31-4ff5-49a4-a884-fbc3a359e1be None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:13:32.495922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:32.497076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:32.504850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:13:32.504850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:13:32.504850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:13:32.504850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:13:32.504850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:13:32.504850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:13:32.504850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:32.504850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:32.504850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 624/1258] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:32.505688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa978-1e9b-411e-81f1-349a7199d1db None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.622825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eefa978-1e9b-411e-81f1-349a7199d1db None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.661708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a9747b-573d-4dca-baa8-82373be1f182 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.662405 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9747b-573d-4dca-baa8-82373be1f182 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.662764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9747b-573d-4dca-baa8-82373be1f182 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.663214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a9747b-573d-4dca-baa8-82373be1f182 None admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.664258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9747b-573d-4dca-baa8-82373be1f182 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:32.664684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9747b-573d-4dca-baa8-82373be1f182 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.666524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a9747b-573d-4dca-baa8-82373be1f182 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.681051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 635/1259] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:13:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:32.690232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74cb31-4ff5-49a4-a884-fbc3a359e1be None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:32.725010 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:32.812674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a74cb31-4ff5-49a4-a884-fbc3a359e1be None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:32.850002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-446781ab-c369-447b-8cea-bb6a8cc4e069 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:32.850680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446781ab-c369-447b-8cea-bb6a8cc4e069 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:32.850966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446781ab-c369-447b-8cea-bb6a8cc4e069 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:32.851243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-446781ab-c369-447b-8cea-bb6a8cc4e069 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:32.852090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446781ab-c369-447b-8cea-bb6a8cc4e069 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=d4d7142663cb4d1f92ecc4da831ac6b6) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:32.852410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446781ab-c369-447b-8cea-bb6a8cc4e069 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:32.854500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-446781ab-c369-447b-8cea-bb6a8cc4e069 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:32.860918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 625/1260] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:32 2022] GET /identity/v3/role_assignments?user.id=d4d7142663cb4d1f92ecc4da831ac6b6&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 259 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:34.922900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e0587-38d6-4a3a-bb0a-9d506bea5960 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:34.923472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e0587-38d6-4a3a-bb0a-9d506bea5960 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:34.923472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363e0587-38d6-4a3a-bb0a-9d506bea5960 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:34.925757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 636/1261] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:34.931015 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826c0dc-46c6-444b-a91f-b0e158c58332 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:34.931461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826c0dc-46c6-444b-a91f-b0e158c58332 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:34.931572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a826c0dc-46c6-444b-a91f-b0e158c58332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:34.948530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a826c0dc-46c6-444b-a91f-b0e158c58332 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:34.949784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a826c0dc-46c6-444b-a91f-b0e158c58332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:35.030239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 626/1262] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:35.056007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e0587-38d6-4a3a-bb0a-9d506bea5960 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:35.067628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:35.157038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363e0587-38d6-4a3a-bb0a-9d506bea5960 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:35.191114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5420072-885f-4f60-b1c9-ce5035319593 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:35.191819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5420072-885f-4f60-b1c9-ce5035319593 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.192102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5420072-885f-4f60-b1c9-ce5035319593 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.192376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5420072-885f-4f60-b1c9-ce5035319593 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.202152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5420072-885f-4f60-b1c9-ce5035319593 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b5420072-885f-4f60-b1c9-ce5035319593 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:35.210115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:35.211224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:35.212264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 637/1263] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:35.217441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826c0dc-46c6-444b-a91f-b0e158c58332 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:35.236220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:35.313394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a826c0dc-46c6-444b-a91f-b0e158c58332 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:35.349878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2ae6-d953-4f34-a1e0-a92c1fe4011c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:35.350537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2ae6-d953-4f34-a1e0-a92c1fe4011c None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.350817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2ae6-d953-4f34-a1e0-a92c1fe4011c None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.351087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3f2ae6-d953-4f34-a1e0-a92c1fe4011c None admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.351824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f2ae6-d953-4f34-a1e0-a92c1fe4011c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:35.352123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f2ae6-d953-4f34-a1e0-a92c1fe4011c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:35.353481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3f2ae6-d953-4f34-a1e0-a92c1fe4011c None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:35.361333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 627/1264] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:35.367313 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5420072-885f-4f60-b1c9-ce5035319593 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:35.433971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:35.468388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5420072-885f-4f60-b1c9-ce5035319593 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:35.501747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db081d-71f5-4af3-8a3a-fb335caa1cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:35.502522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db081d-71f5-4af3-8a3a-fb335caa1cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.502977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db081d-71f5-4af3-8a3a-fb335caa1cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.503256 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8db081d-71f5-4af3-8a3a-fb335caa1cb1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.514562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db081d-71f5-4af3-8a3a-fb335caa1cb1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:35.516005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8db081d-71f5-4af3-8a3a-fb335caa1cb1 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-b8db081d-71f5-4af3-8a3a-fb335caa1cb1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:35.533012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:35.534711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:35.535910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:35.535910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:35.535910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:35.535910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:35.535910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 638/1265] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:35.540936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2ae6-d953-4f34-a1e0-a92c1fe4011c None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:35.620298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:35.639521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3f2ae6-d953-4f34-a1e0-a92c1fe4011c None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:35.672112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864d85d-7797-47f0-9af2-1644578ad014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:35.672771 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d85d-7797-47f0-9af2-1644578ad014 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.673059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d85d-7797-47f0-9af2-1644578ad014 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.673339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7864d85d-7797-47f0-9af2-1644578ad014 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.674089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864d85d-7797-47f0-9af2-1644578ad014 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:35.674451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864d85d-7797-47f0-9af2-1644578ad014 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:35.675836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7864d85d-7797-47f0-9af2-1644578ad014 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:35.687231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 628/1266] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:35.693602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db081d-71f5-4af3-8a3a-fb335caa1cb1 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:35.792823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8db081d-71f5-4af3-8a3a-fb335caa1cb1 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:35.812373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:35.826490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826cad6-0154-44a4-8286-3db0258fc034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:35.827278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826cad6-0154-44a4-8286-3db0258fc034 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.827561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826cad6-0154-44a4-8286-3db0258fc034 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.827835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0826cad6-0154-44a4-8286-3db0258fc034 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.836565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826cad6-0154-44a4-8286-3db0258fc034 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:35.838012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0826cad6-0154-44a4-8286-3db0258fc034 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-0826cad6-0154-44a4-8286-3db0258fc034 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:13:35.852412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:35.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:35.854602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 639/1267] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/users/placement => generated 87 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:35.859594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864d85d-7797-47f0-9af2-1644578ad014 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:35.959650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7864d85d-7797-47f0-9af2-1644578ad014 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:35.995562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008da138-16f5-4b45-98bf-3cfbe8663d57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:35.996319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008da138-16f5-4b45-98bf-3cfbe8663d57 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:35.996614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008da138-16f5-4b45-98bf-3cfbe8663d57 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:35.996987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-008da138-16f5-4b45-98bf-3cfbe8663d57 None admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:35.997870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008da138-16f5-4b45-98bf-3cfbe8663d57 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:35.998229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008da138-16f5-4b45-98bf-3cfbe8663d57 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:35.999962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-008da138-16f5-4b45-98bf-3cfbe8663d57 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:36.011591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:36.022528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 629/1268] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Jul 20 19:13:35 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:36.029225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826cad6-0154-44a4-8286-3db0258fc034 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:36.128415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0826cad6-0154-44a4-8286-3db0258fc034 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:36.161695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aafad4-f44b-41fd-a798-f6e975e29857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:36.162471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafad4-f44b-41fd-a798-f6e975e29857 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:36.162755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafad4-f44b-41fd-a798-f6e975e29857 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:36.163030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5aafad4-f44b-41fd-a798-f6e975e29857 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:36.173339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aafad4-f44b-41fd-a798-f6e975e29857 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:36.174879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5aafad4-f44b-41fd-a798-f6e975e29857 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-f5aafad4-f44b-41fd-a798-f6e975e29857 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:36.191473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:36.192757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:36.194318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:36.194318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:36.194318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:36.194318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:36.194318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 640/1269] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:36.198386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008da138-16f5-4b45-98bf-3cfbe8663d57 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:36.296656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-008da138-16f5-4b45-98bf-3cfbe8663d57 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:36.330124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851918f-feaf-4be0-a6bb-07aa6851db80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:36.331331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851918f-feaf-4be0-a6bb-07aa6851db80 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:36.331617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851918f-feaf-4be0-a6bb-07aa6851db80 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:36.331890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6851918f-feaf-4be0-a6bb-07aa6851db80 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:36.332604 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851918f-feaf-4be0-a6bb-07aa6851db80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:36.332912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851918f-feaf-4be0-a6bb-07aa6851db80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:36.334316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6851918f-feaf-4be0-a6bb-07aa6851db80 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:36.345297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 630/1270] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:36.351844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aafad4-f44b-41fd-a798-f6e975e29857 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:36.382982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:36.449240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5aafad4-f44b-41fd-a798-f6e975e29857 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:36.481825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309f129-d396-4ced-9b53-da8fea362505 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:36.482602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309f129-d396-4ced-9b53-da8fea362505 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:36.482887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309f129-d396-4ced-9b53-da8fea362505 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:36.483161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3309f129-d396-4ced-9b53-da8fea362505 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:36.490280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309f129-d396-4ced-9b53-da8fea362505 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:36.491826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3309f129-d396-4ced-9b53-da8fea362505 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-3309f129-d396-4ced-9b53-da8fea362505 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:13:36.506284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:36.507359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:36.508325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:13:36.508325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:13:36.508325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:13:36.508325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:13:36.508325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:13:36.508325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:13:36.508325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:36.508325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:36.508325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 641/1271] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:36.513316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851918f-feaf-4be0-a6bb-07aa6851db80 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:36.570928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:36.611770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6851918f-feaf-4be0-a6bb-07aa6851db80 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:36.644843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dfc8e12-305b-450d-8e05-247e7005a8bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:36.645431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc8e12-305b-450d-8e05-247e7005a8bf None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:36.645787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc8e12-305b-450d-8e05-247e7005a8bf None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:36.646023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfc8e12-305b-450d-8e05-247e7005a8bf None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:36.646802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc8e12-305b-450d-8e05-247e7005a8bf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:36.647112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc8e12-305b-450d-8e05-247e7005a8bf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:36.648542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dfc8e12-305b-450d-8e05-247e7005a8bf None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:36.668091 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 631/1272] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:13:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:36.674163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309f129-d396-4ced-9b53-da8fea362505 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:36.764350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:36.774169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3309f129-d396-4ced-9b53-da8fea362505 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:36.807407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97332007-068b-437e-979d-19d38067a5f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:36.808281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332007-068b-437e-979d-19d38067a5f2 None admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:36.808565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332007-068b-437e-979d-19d38067a5f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:36.808841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97332007-068b-437e-979d-19d38067a5f2 None admin] PATH_INFO: `/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/d4d7142663cb4d1f92ecc4da831ac6b6/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:36.810805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97332007-068b-437e-979d-19d38067a5f2 None admin] RBAC: Authorizing `identity:create_grant(project_id=2777f7eea4c94b78af6eda0a1e703617, user_id=d4d7142663cb4d1f92ecc4da831ac6b6, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:36.812232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97332007-068b-437e-979d-19d38067a5f2 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:36.825206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 642/1273] 10.222.0.69 () {58 vars in 1439 bytes} [Wed Jul 20 19:13:36 2022] PUT /identity/v3/projects/2777f7eea4c94b78af6eda0a1e703617/users/d4d7142663cb4d1f92ecc4da831ac6b6/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:13:38.873904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed570-b56a-47aa-a7e7-fdb1d0194b0f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:38.874450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed570-b56a-47aa-a7e7-fdb1d0194b0f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:38.874450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32ed570-b56a-47aa-a7e7-fdb1d0194b0f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:38.875834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 632/1274] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:38.881341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6f43e-6152-4d56-9213-7476f7a032f1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:38.881823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6f43e-6152-4d56-9213-7476f7a032f1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:38.881936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae6f43e-6152-4d56-9213-7476f7a032f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:38.899764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-eae6f43e-6152-4d56-9213-7476f7a032f1 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:38.900954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-eae6f43e-6152-4d56-9213-7476f7a032f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:38.974626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 643/1275] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:38.997999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed570-b56a-47aa-a7e7-fdb1d0194b0f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:39.110750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32ed570-b56a-47aa-a7e7-fdb1d0194b0f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:39.128749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:39.146488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759eee2-cafa-4ce0-b437-d3e9216a9764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:39.147289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759eee2-cafa-4ce0-b437-d3e9216a9764 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:39.147576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759eee2-cafa-4ce0-b437-d3e9216a9764 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:39.147852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7759eee2-cafa-4ce0-b437-d3e9216a9764 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:39.158000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7759eee2-cafa-4ce0-b437-d3e9216a9764 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-7759eee2-cafa-4ce0-b437-d3e9216a9764 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 20 19:13:39.166384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:39.167590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:39.168516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 633/1276] 10.222.0.69 () {56 vars in 1096 bytes} [Wed Jul 20 19:13:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 20 19:13:39.173635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6f43e-6152-4d56-9213-7476f7a032f1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:39.269994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:39.291665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae6f43e-6152-4d56-9213-7476f7a032f1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:39.329332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df4fd4-2e49-467a-81da-328443d57298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:39.330026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df4fd4-2e49-467a-81da-328443d57298 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:39.330307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df4fd4-2e49-467a-81da-328443d57298 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:39.330628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36df4fd4-2e49-467a-81da-328443d57298 None admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:39.331489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df4fd4-2e49-467a-81da-328443d57298 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:39.331801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df4fd4-2e49-467a-81da-328443d57298 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:39.333154 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36df4fd4-2e49-467a-81da-328443d57298 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:39.341480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 644/1277] 10.222.0.69 () {56 vars in 1099 bytes} [Wed Jul 20 19:13:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:39.348076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759eee2-cafa-4ce0-b437-d3e9216a9764 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:39.451169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7759eee2-cafa-4ce0-b437-d3e9216a9764 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:39.485399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d22648-c62c-4e3a-afc1-66d35f947fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:39.486207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d22648-c62c-4e3a-afc1-66d35f947fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:39.486490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d22648-c62c-4e3a-afc1-66d35f947fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:39.486833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d22648-c62c-4e3a-afc1-66d35f947fd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:39.499152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d22648-c62c-4e3a-afc1-66d35f947fd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:39.500581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d22648-c62c-4e3a-afc1-66d35f947fd3 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-a2d22648-c62c-4e3a-afc1-66d35f947fd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:39.517693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:39.518808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:39.519808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:39.519808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:39.519808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:39.519808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:39.519808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 634/1278] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:39.525161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df4fd4-2e49-467a-81da-328443d57298 None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:39.624980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36df4fd4-2e49-467a-81da-328443d57298 None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:39.657791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587bc25-d2b5-4464-a24f-1b8dea0fd5ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:39.658448 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587bc25-d2b5-4464-a24f-1b8dea0fd5ad None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:39.659302 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587bc25-d2b5-4464-a24f-1b8dea0fd5ad None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:39.659685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8587bc25-d2b5-4464-a24f-1b8dea0fd5ad None admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:39.660541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587bc25-d2b5-4464-a24f-1b8dea0fd5ad None admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:39.660854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587bc25-d2b5-4464-a24f-1b8dea0fd5ad None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:39.662239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8587bc25-d2b5-4464-a24f-1b8dea0fd5ad None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:39.673768 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 645/1279] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:39.680171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d22648-c62c-4e3a-afc1-66d35f947fd3 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:39.716628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:39.789811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d22648-c62c-4e3a-afc1-66d35f947fd3 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:39.824191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7b28d-a8d5-453d-97aa-2c40705ce4a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:39.824994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7b28d-a8d5-453d-97aa-2c40705ce4a7 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:39.825278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7b28d-a8d5-453d-97aa-2c40705ce4a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:39.825579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed7b28d-a8d5-453d-97aa-2c40705ce4a7 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:39.834041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7b28d-a8d5-453d-97aa-2c40705ce4a7 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:39.835476 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed7b28d-a8d5-453d-97aa-2c40705ce4a7 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-0ed7b28d-a8d5-453d-97aa-2c40705ce4a7 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 20 19:13:39.850590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:39.851753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:39.852941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 635/1280] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:39 2022] GET /identity/v3/users/placement => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:39.857756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587bc25-d2b5-4464-a24f-1b8dea0fd5ad None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:39.863151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:39.962135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8587bc25-d2b5-4464-a24f-1b8dea0fd5ad None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:40.002441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0706236-4a62-46f9-89a8-bcdcd867f0fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:40.003361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0706236-4a62-46f9-89a8-bcdcd867f0fd None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:40.003547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0706236-4a62-46f9-89a8-bcdcd867f0fd None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:40.003929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0706236-4a62-46f9-89a8-bcdcd867f0fd None admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:40.004835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0706236-4a62-46f9-89a8-bcdcd867f0fd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:40.005148 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0706236-4a62-46f9-89a8-bcdcd867f0fd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:40.006809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0706236-4a62-46f9-89a8-bcdcd867f0fd None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:40.029046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 646/1281] 10.222.0.69 () {56 vars in 1143 bytes} [Wed Jul 20 19:13:39 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:40.036870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7b28d-a8d5-453d-97aa-2c40705ce4a7 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:40.134899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:40.145834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed7b28d-a8d5-453d-97aa-2c40705ce4a7 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:40.182601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95a35e-deba-4288-9914-a4449b033bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:40.183258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95a35e-deba-4288-9914-a4449b033bce None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:40.183544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95a35e-deba-4288-9914-a4449b033bce None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:40.183820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95a35e-deba-4288-9914-a4449b033bce None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:40.254445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95a35e-deba-4288-9914-a4449b033bce None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:40.255818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95a35e-deba-4288-9914-a4449b033bce None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:40.355491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:40.366252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 636/1282] 10.222.0.69 () {58 vars in 1282 bytes} [Wed Jul 20 19:13:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 20 19:13:40.372136 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0706236-4a62-46f9-89a8-bcdcd867f0fd None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:40.435931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:40.477393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0706236-4a62-46f9-89a8-bcdcd867f0fd None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:40.518498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e245c-6263-4ffe-8b32-430597b4519b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:40.519193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e245c-6263-4ffe-8b32-430597b4519b None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:40.519480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e245c-6263-4ffe-8b32-430597b4519b None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:40.519760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-362e245c-6263-4ffe-8b32-430597b4519b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:40.535072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e245c-6263-4ffe-8b32-430597b4519b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:40.536555 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-362e245c-6263-4ffe-8b32-430597b4519b None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-362e245c-6263-4ffe-8b32-430597b4519b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 20 19:13:40.555710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:40.560484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:40.562106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 20 19:13:40.562106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 20 19:13:40.562106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 20 19:13:40.562106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:40.562106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 647/1283] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:13:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:40.562772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95a35e-deba-4288-9914-a4449b033bce None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:40.667685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95a35e-deba-4288-9914-a4449b033bce None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:40.707073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a90ae-6d12-460a-be3e-0f2e26ad4ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:40.707749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a90ae-6d12-460a-be3e-0f2e26ad4ce4 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:40.708030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a90ae-6d12-460a-be3e-0f2e26ad4ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:40.708310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d72a90ae-6d12-460a-be3e-0f2e26ad4ce4 None admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:40.709138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a90ae-6d12-460a-be3e-0f2e26ad4ce4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:40.709348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a90ae-6d12-460a-be3e-0f2e26ad4ce4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:40.710760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d72a90ae-6d12-460a-be3e-0f2e26ad4ce4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:40.722717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 637/1284] 10.222.0.69 () {56 vars in 1109 bytes} [Wed Jul 20 19:13:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:40.728390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e245c-6263-4ffe-8b32-430597b4519b None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:40.827477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-362e245c-6263-4ffe-8b32-430597b4519b None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:40.841240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:40.862596 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f012c8-84c4-4048-b9df-05bc0297151c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:40.863389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f012c8-84c4-4048-b9df-05bc0297151c None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:40.863771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f012c8-84c4-4048-b9df-05bc0297151c None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:40.863952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f012c8-84c4-4048-b9df-05bc0297151c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:40.870702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f012c8-84c4-4048-b9df-05bc0297151c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:40.872136 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f012c8-84c4-4048-b9df-05bc0297151c None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application [None req-15f012c8-84c4-4048-b9df-05bc0297151c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 20 19:13:40.885609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:40.886692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:40.887880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 20 19:13:40.887880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 20 19:13:40.887880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 20 19:13:40.887880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 20 19:13:40.887880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 20 19:13:40.887880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 20 19:13:40.887880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 20 19:13:40.887880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: ERROR keystone.server.flask.application Jul 20 19:13:40.887880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 648/1285] 10.222.0.69 () {56 vars in 1111 bytes} [Wed Jul 20 19:13:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:40.892681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a90ae-6d12-460a-be3e-0f2e26ad4ce4 None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:40.942582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:40.997091 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d72a90ae-6d12-460a-be3e-0f2e26ad4ce4 None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:41.030803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d242c9-ab26-422b-afb6-c81a77d6fb1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:41.031458 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d242c9-ab26-422b-afb6-c81a77d6fb1d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:41.031741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d242c9-ab26-422b-afb6-c81a77d6fb1d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:41.032017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d242c9-ab26-422b-afb6-c81a77d6fb1d None admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:41.032790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d242c9-ab26-422b-afb6-c81a77d6fb1d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:41.033101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d242c9-ab26-422b-afb6-c81a77d6fb1d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:41.035219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d242c9-ab26-422b-afb6-c81a77d6fb1d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:41.046852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 638/1286] 10.222.0.69 () {56 vars in 1148 bytes} [Wed Jul 20 19:13:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:41.053989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f012c8-84c4-4048-b9df-05bc0297151c None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:41.150719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f012c8-84c4-4048-b9df-05bc0297151c None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:41.183567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4f1251-c653-4449-b5ec-053630c9d253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:41.184200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f1251-c653-4449-b5ec-053630c9d253 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:41.184481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f1251-c653-4449-b5ec-053630c9d253 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:41.184756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4f1251-c653-4449-b5ec-053630c9d253 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:41.185595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f1251-c653-4449-b5ec-053630c9d253 None admin] RBAC: Adding query filter params (role.id=58ca114197894385aca5fa8e7cf9f4c7, scope.project.id=2777f7eea4c94b78af6eda0a1e703617, user.id=d4d7142663cb4d1f92ecc4da831ac6b6) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:41.185905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f1251-c653-4449-b5ec-053630c9d253 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:41.187310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4f1251-c653-4449-b5ec-053630c9d253 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:41.193241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 649/1287] 10.222.0.69 () {56 vars in 1374 bytes} [Wed Jul 20 19:13:41 2022] GET /identity/v3/role_assignments?user.id=d4d7142663cb4d1f92ecc4da831ac6b6&scope.project.id=2777f7eea4c94b78af6eda0a1e703617&role.id=58ca114197894385aca5fa8e7cf9f4c7 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:43.259432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c957a-67b8-4957-b084-abe486e3233c None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.259995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c957a-67b8-4957-b084-abe486e3233c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.259995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361c957a-67b8-4957-b084-abe486e3233c None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.261352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 639/1288] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:13:43.266497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71535b03-dd49-437f-af19-c7c5a91b99e1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.266869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71535b03-dd49-437f-af19-c7c5a91b99e1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.267051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71535b03-dd49-437f-af19-c7c5a91b99e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.283465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-71535b03-dd49-437f-af19-c7c5a91b99e1 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:43.284734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-71535b03-dd49-437f-af19-c7c5a91b99e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:43.314150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:43.353255 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 650/1289] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:43.375699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c957a-67b8-4957-b084-abe486e3233c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.482683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361c957a-67b8-4957-b084-abe486e3233c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.493673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:43.516705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000bb3d-d0d8-47e2-83d1-879c6f466fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.517425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000bb3d-d0d8-47e2-83d1-879c6f466fee None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.517773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000bb3d-d0d8-47e2-83d1-879c6f466fee None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.518069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c000bb3d-d0d8-47e2-83d1-879c6f466fee None admin] PATH_INFO: `/v3/services/placement` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.518716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000bb3d-d0d8-47e2-83d1-879c6f466fee None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:43.520084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c000bb3d-d0d8-47e2-83d1-879c6f466fee None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-c000bb3d-d0d8-47e2-83d1-879c6f466fee None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:43.532029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:43.533300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:43.535088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 640/1290] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:43.539995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71535b03-dd49-437f-af19-c7c5a91b99e1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.637096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71535b03-dd49-437f-af19-c7c5a91b99e1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.670571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d74a88-2cef-4b53-ac4c-27256498a669 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.671220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d74a88-2cef-4b53-ac4c-27256498a669 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.671503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d74a88-2cef-4b53-ac4c-27256498a669 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.671778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d74a88-2cef-4b53-ac4c-27256498a669 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.672478 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d74a88-2cef-4b53-ac4c-27256498a669 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:43.672786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d74a88-2cef-4b53-ac4c-27256498a669 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:43.674739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d74a88-2cef-4b53-ac4c-27256498a669 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:43.679536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 651/1291] 10.222.0.69 () {56 vars in 1116 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:43.685374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000bb3d-d0d8-47e2-83d1-879c6f466fee None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:43.783428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c000bb3d-d0d8-47e2-83d1-879c6f466fee None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:43.815818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc185169-6dcf-4fff-917c-bd91fb0b15b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:43.816463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185169-6dcf-4fff-917c-bd91fb0b15b1 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:43.816748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185169-6dcf-4fff-917c-bd91fb0b15b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:43.817026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc185169-6dcf-4fff-917c-bd91fb0b15b1 None admin] PATH_INFO: `/v3/services` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:43.817755 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185169-6dcf-4fff-917c-bd91fb0b15b1 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:43.818065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185169-6dcf-4fff-917c-bd91fb0b15b1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:43.819440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc185169-6dcf-4fff-917c-bd91fb0b15b1 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:43.823797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 641/1292] 10.222.0.69 () {56 vars in 1116 bytes} [Wed Jul 20 19:13:43 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:45.860841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198d68a-d517-405e-83ee-4d9bb74e0e30 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.861427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198d68a-d517-405e-83ee-4d9bb74e0e30 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.862092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0198d68a-d517-405e-83ee-4d9bb74e0e30 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.863500 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 652/1293] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:45.868725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60735d-bc2a-4f44-bba0-6fb51f697f89 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:45.869140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60735d-bc2a-4f44-bba0-6fb51f697f89 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:45.869280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc60735d-bc2a-4f44-bba0-6fb51f697f89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:45.885710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cc60735d-bc2a-4f44-bba0-6fb51f697f89 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:45.886879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cc60735d-bc2a-4f44-bba0-6fb51f697f89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:45.918862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:45.955886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 642/1294] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 20 19:13:45.979325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198d68a-d517-405e-83ee-4d9bb74e0e30 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:46.014896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:46.104216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0198d68a-d517-405e-83ee-4d9bb74e0e30 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:46.137593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21e4cf-6900-4f64-8b9f-0023e7e204fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:46.138193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21e4cf-6900-4f64-8b9f-0023e7e204fb None admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:46.138481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21e4cf-6900-4f64-8b9f-0023e7e204fb None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:46.138759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21e4cf-6900-4f64-8b9f-0023e7e204fb None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:46.139494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21e4cf-6900-4f64-8b9f-0023e7e204fb None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:46.140858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21e4cf-6900-4f64-8b9f-0023e7e204fb None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:46.154403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 653/1295] 10.222.0.69 () {60 vars in 1141 bytes} [Wed Jul 20 19:13:45 2022] POST /identity/v3/services => generated 246 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:13:48.180687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4da8f-36ae-456b-9e63-f5b08a41def8 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.181223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4da8f-36ae-456b-9e63-f5b08a41def8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.181223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feb4da8f-36ae-456b-9e63-f5b08a41def8 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.182671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 643/1296] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:48.187747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1192e-0b06-4512-81d2-5be027a8957e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.188122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1192e-0b06-4512-81d2-5be027a8957e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.188300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec1192e-0b06-4512-81d2-5be027a8957e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.205877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-eec1192e-0b06-4512-81d2-5be027a8957e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:48.207124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-eec1192e-0b06-4512-81d2-5be027a8957e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:48.281489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 654/1297] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:48 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:48.303997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4da8f-36ae-456b-9e63-f5b08a41def8 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:48.405984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feb4da8f-36ae-456b-9e63-f5b08a41def8 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.439270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d38f1-6fff-44f1-a109-a800629f4e1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:48.440681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d38f1-6fff-44f1-a109-a800629f4e1e None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.440957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d38f1-6fff-44f1-a109-a800629f4e1e None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.441230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541d38f1-6fff-44f1-a109-a800629f4e1e None admin] PATH_INFO: `/v3/services/placement` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.441922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d38f1-6fff-44f1-a109-a800629f4e1e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:48.443286 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541d38f1-6fff-44f1-a109-a800629f4e1e None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-541d38f1-6fff-44f1-a109-a800629f4e1e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:48.454980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:48.456273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:48.457463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 644/1298] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Jul 20 19:13:48 2022] GET /identity/v3/services/placement => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:48.462295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1192e-0b06-4512-81d2-5be027a8957e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:48.481036 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:48.560006 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec1192e-0b06-4512-81d2-5be027a8957e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.592502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a172b-2703-4a17-878c-fac5ed7143ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:48.593153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a172b-2703-4a17-878c-fac5ed7143ce None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.593431 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a172b-2703-4a17-878c-fac5ed7143ce None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.593727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143a172b-2703-4a17-878c-fac5ed7143ce None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.594420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a172b-2703-4a17-878c-fac5ed7143ce None admin] RBAC: Adding query filter params (name=placement) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:48.594723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a172b-2703-4a17-878c-fac5ed7143ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:48.596335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143a172b-2703-4a17-878c-fac5ed7143ce None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:48.601408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 655/1299] 10.222.0.69 () {56 vars in 1116 bytes} [Wed Jul 20 19:13:48 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:48.607344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d38f1-6fff-44f1-a109-a800629f4e1e None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:48.638419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:48.704084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541d38f1-6fff-44f1-a109-a800629f4e1e None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.737035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29b66b0d-021e-446e-8da5-c335a51fa307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:48.737706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b66b0d-021e-446e-8da5-c335a51fa307 None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.737986 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b66b0d-021e-446e-8da5-c335a51fa307 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.738252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29b66b0d-021e-446e-8da5-c335a51fa307 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.739020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b66b0d-021e-446e-8da5-c335a51fa307 None admin] RBAC: Adding query filter params (interface=public, service_id=83576299d09e4da1abeca4dcaa4a189d, region_id=RegionOne) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:48.739323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b66b0d-021e-446e-8da5-c335a51fa307 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:48.740679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29b66b0d-021e-446e-8da5-c335a51fa307 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:48.745581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 645/1300] 10.222.0.69 () {56 vars in 1251 bytes} [Wed Jul 20 19:13:48 2022] GET /identity/v3/endpoints?service_id=83576299d09e4da1abeca4dcaa4a189d&interface=public®ion_id=RegionOne => generated 194 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:48.751219 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a172b-2703-4a17-878c-fac5ed7143ce None admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:48.832061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:48.850580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143a172b-2703-4a17-878c-fac5ed7143ce None admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:48.883798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fb1921-a719-496e-948c-722977a93268 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:48.884453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb1921-a719-496e-948c-722977a93268 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:48.884739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb1921-a719-496e-948c-722977a93268 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:48.885015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fb1921-a719-496e-948c-722977a93268 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:48.885722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb1921-a719-496e-948c-722977a93268 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:48.887081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fb1921-a719-496e-948c-722977a93268 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:48.893842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 656/1301] 10.222.0.69 () {56 vars in 1087 bytes} [Wed Jul 20 19:13:48 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:13:50.910600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da3b58-de78-4f4e-a48d-d8670956e9be None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.911186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da3b58-de78-4f4e-a48d-d8670956e9be None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.911186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8da3b58-de78-4f4e-a48d-d8670956e9be None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.912603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 646/1302] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:13:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:13:50.918385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509e3f9-4d61-4bd6-85f2-e4ef929687e4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:50.918789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509e3f9-4d61-4bd6-85f2-e4ef929687e4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:50.918941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9509e3f9-4d61-4bd6-85f2-e4ef929687e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:50.936039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9509e3f9-4d61-4bd6-85f2-e4ef929687e4 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:13:50.937223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9509e3f9-4d61-4bd6-85f2-e4ef929687e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:13:51.012250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 657/1303] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:13:50 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 20 19:13:51.036141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da3b58-de78-4f4e-a48d-d8670956e9be None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:51.135077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:51.143388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8da3b58-de78-4f4e-a48d-d8670956e9be None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:51.177059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30572083-fd7c-4e47-8fd9-8435757dbb9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:51.177818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30572083-fd7c-4e47-8fd9-8435757dbb9d None admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:51.178105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30572083-fd7c-4e47-8fd9-8435757dbb9d None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:51.178384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30572083-fd7c-4e47-8fd9-8435757dbb9d None admin] PATH_INFO: `/v3/services/placement` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:51.179032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30572083-fd7c-4e47-8fd9-8435757dbb9d None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:51.180404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30572083-fd7c-4e47-8fd9-8435757dbb9d None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application [None req-30572083-fd7c-4e47-8fd9-8435757dbb9d None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 20 19:13:51.192738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 20 19:13:51.193936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.get_or_create( Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application with Lock( Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self._enter() Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application return self.creator() Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application created_value = creator( Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: ERROR keystone.server.flask.application Jul 20 19:13:51.194845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 647/1304] 10.222.0.69 () {56 vars in 1117 bytes} [Wed Jul 20 19:13:51 2022] GET /identity/v3/services/placement => generated 90 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 20 19:13:51.200380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509e3f9-4d61-4bd6-85f2-e4ef929687e4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:51.277642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:13:51.301160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9509e3f9-4d61-4bd6-85f2-e4ef929687e4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:51.335824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fd7e49-aa79-4c99-95c4-1f4b546e9613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:51.336487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd7e49-aa79-4c99-95c4-1f4b546e9613 None admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:51.336769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd7e49-aa79-4c99-95c4-1f4b546e9613 None admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:51.337045 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fd7e49-aa79-4c99-95c4-1f4b546e9613 None admin] PATH_INFO: `/v3/services` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:51.337778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd7e49-aa79-4c99-95c4-1f4b546e9613 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:13:51.338088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd7e49-aa79-4c99-95c4-1f4b546e9613 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:51.339455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fd7e49-aa79-4c99-95c4-1f4b546e9613 None admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:51.344602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 658/1305] 10.222.0.69 () {56 vars in 1116 bytes} [Wed Jul 20 19:13:51 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:13:51.350671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30572083-fd7c-4e47-8fd9-8435757dbb9d None admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:13:51.452165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30572083-fd7c-4e47-8fd9-8435757dbb9d None admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:13:51.484892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8099c3c-d3d0-4004-b964-a2e4218915f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:13:51.485571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8099c3c-d3d0-4004-b964-a2e4218915f4 None admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:13:51.485854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8099c3c-d3d0-4004-b964-a2e4218915f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:13:51.486698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8099c3c-d3d0-4004-b964-a2e4218915f4 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:13:51.487555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8099c3c-d3d0-4004-b964-a2e4218915f4 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:13:51.488920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8099c3c-d3d0-4004-b964-a2e4218915f4 None admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:13:51.507219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 648/1306] 10.222.0.69 () {60 vars in 1144 bytes} [Wed Jul 20 19:13:51 2022] POST /identity/v3/endpoints => generated 331 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:14:01.341846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=30 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:01.502481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:56.509402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45961ee1-d5e7-49d2-b58b-6891e2d6e736 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.509930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45961ee1-d5e7-49d2-b58b-6891e2d6e736 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.510182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45961ee1-d5e7-49d2-b58b-6891e2d6e736 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.511718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 659/1307] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:14:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:56.517105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc93e9-40d3-4c86-9b24-cf8625fd1fe6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.517526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc93e9-40d3-4c86-9b24-cf8625fd1fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.517776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bc93e9-40d3-4c86-9b24-cf8625fd1fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.644156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-40bc93e9-40d3-4c86-9b24-cf8625fd1fe6 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:56.645415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-40bc93e9-40d3-4c86-9b24-cf8625fd1fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:56.785588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 649/1308] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:14:56.828032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceba15d-b701-453e-ab67-b978431dedbd None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.828327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceba15d-b701-453e-ab67-b978431dedbd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.828602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ceba15d-b701-453e-ab67-b978431dedbd None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.830327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 660/1309] 10.222.0.69 () {54 vars in 965 bytes} [Wed Jul 20 19:14:56 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:56.836798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff64c6-0686-4f78-b387-9f1240c8f987 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:56.837094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff64c6-0686-4f78-b387-9f1240c8f987 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:56.837467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ff64c6-0686-4f78-b387-9f1240c8f987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:56.927383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d4ff64c6-0686-4f78-b387-9f1240c8f987 None None] MFA Rules not processed for user `3a1a88089d384964bd9774eda4c5d8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:56.928600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d4ff64c6-0686-4f78-b387-9f1240c8f987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:56.953185 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:57.034980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceba15d-b701-453e-ab67-b978431dedbd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:14:57.036732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5ceba15d-b701-453e-ab67-b978431dedbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:57.046778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 650/1310] 10.222.0.69 () {58 vars in 1062 bytes} [Wed Jul 20 19:14:56 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:14:57.176337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ceba15d-b701-453e-ab67-b978431dedbd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:14:57.270861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c7662f-bf86-4b59-96b1-e8db66820f23 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:14:57.271557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7662f-bf86-4b59-96b1-e8db66820f23 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:57.271844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7662f-bf86-4b59-96b1-e8db66820f23 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:57.272124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c7662f-bf86-4b59-96b1-e8db66820f23 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:57.274486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e4c7662f-bf86-4b59-96b1-e8db66820f23 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:14:57.345699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:14:57.411898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7662f-bf86-4b59-96b1-e8db66820f23 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:14:57.413364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c7662f-bf86-4b59-96b1-e8db66820f23 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:14:57.447678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 661/1311] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:14:59.896394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072b47e-0a11-4799-a415-d887b70da969 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:59.896985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072b47e-0a11-4799-a415-d887b70da969 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:59.896985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072b47e-0a11-4799-a415-d887b70da969 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:59.898399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 651/1312] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:14:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:14:59.903979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693453-9c1d-4b04-aca3-98cf68da62dc None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:14:59.905065 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693453-9c1d-4b04-aca3-98cf68da62dc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:14:59.905249 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08693453-9c1d-4b04-aca3-98cf68da62dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:14:59.985109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-08693453-9c1d-4b04-aca3-98cf68da62dc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:14:59.986527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-08693453-9c1d-4b04-aca3-98cf68da62dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:00.085943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 662/1313] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:00.124807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072b47e-0a11-4799-a415-d887b70da969 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:00.126483 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e072b47e-0a11-4799-a415-d887b70da969 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:00.237355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072b47e-0a11-4799-a415-d887b70da969 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:00.311456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:00.320344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f4b9cf5-eb11-469f-8107-8b7deb69b564 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:00.321417 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b9cf5-eb11-469f-8107-8b7deb69b564 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:00.321417 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b9cf5-eb11-469f-8107-8b7deb69b564 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:00.321688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f4b9cf5-eb11-469f-8107-8b7deb69b564 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:00.324050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1f4b9cf5-eb11-469f-8107-8b7deb69b564 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:00.439427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b9cf5-eb11-469f-8107-8b7deb69b564 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:00.440890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f4b9cf5-eb11-469f-8107-8b7deb69b564 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:00.460036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 652/1314] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:02.944225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedd6d6-3366-4115-b143-4c58779b8bbe None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:02.944882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedd6d6-3366-4115-b143-4c58779b8bbe None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:02.945279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bedd6d6-3366-4115-b143-4c58779b8bbe None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:02.948266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 663/1315] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:02.953967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52fa1-86b1-41f2-8728-ded6b275ef53 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:02.954428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52fa1-86b1-41f2-8728-ded6b275ef53 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:02.954661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b52fa1-86b1-41f2-8728-ded6b275ef53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.021060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-87b52fa1-86b1-41f2-8728-ded6b275ef53 None None] MFA Rules not processed for user `343f2a00f3ed43acb00bd614cb69766e`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:03.022296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-87b52fa1-86b1-41f2-8728-ded6b275ef53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:03.073802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:03.119359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 653/1316] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:02 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:03.160600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedd6d6-3366-4115-b143-4c58779b8bbe None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:03.162195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0bedd6d6-3366-4115-b143-4c58779b8bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:03.171770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:03.272333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bedd6d6-3366-4115-b143-4c58779b8bbe None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:03.344598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79400f63-375e-4fd6-8232-714c78d52ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:03.345497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79400f63-375e-4fd6-8232-714c78d52ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:03.345805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79400f63-375e-4fd6-8232-714c78d52ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:03.346078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79400f63-375e-4fd6-8232-714c78d52ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:03.347730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-79400f63-375e-4fd6-8232-714c78d52ab1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:03.411115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:03.458736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79400f63-375e-4fd6-8232-714c78d52ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:03.460172 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79400f63-375e-4fd6-8232-714c78d52ab1 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:03.478948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 664/1317] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:03 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:15:06.182139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcf6ca-762c-4b77-bfeb-ed10b70c2fc0 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.182631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcf6ca-762c-4b77-bfeb-ed10b70c2fc0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.182741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37dcf6ca-762c-4b77-bfeb-ed10b70c2fc0 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.184068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 654/1318] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:06.189905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebaa95-b83f-4d3f-981e-396b97f55b0c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.190337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebaa95-b83f-4d3f-981e-396b97f55b0c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.190488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbebaa95-b83f-4d3f-981e-396b97f55b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.254769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bbebaa95-b83f-4d3f-981e-396b97f55b0c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:06.255974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bbebaa95-b83f-4d3f-981e-396b97f55b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:06.355272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 665/1319] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:06 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:06.395519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcf6ca-762c-4b77-bfeb-ed10b70c2fc0 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:06.397104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-37dcf6ca-762c-4b77-bfeb-ed10b70c2fc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:06.510155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37dcf6ca-762c-4b77-bfeb-ed10b70c2fc0 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:06.583480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9207e6c7-c593-4899-94eb-0f352d86af0b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:06.584125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207e6c7-c593-4899-94eb-0f352d86af0b service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:06.584405 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207e6c7-c593-4899-94eb-0f352d86af0b service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:06.584858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9207e6c7-c593-4899-94eb-0f352d86af0b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:06.586344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9207e6c7-c593-4899-94eb-0f352d86af0b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:06.594258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:06.703407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207e6c7-c593-4899-94eb-0f352d86af0b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:06.704868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9207e6c7-c593-4899-94eb-0f352d86af0b service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:06.723960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 655/1320] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:06 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:09.472772 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862af0-6ae3-41de-b176-04dc23067016 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.473313 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862af0-6ae3-41de-b176-04dc23067016 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.473313 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48862af0-6ae3-41de-b176-04dc23067016 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.474723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 666/1321] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:09.479895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cd429-5921-4c8c-947c-960e3d1da39b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.480306 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cd429-5921-4c8c-947c-960e3d1da39b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.480553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a50cd429-5921-4c8c-947c-960e3d1da39b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.549082 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a50cd429-5921-4c8c-947c-960e3d1da39b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:09.550339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a50cd429-5921-4c8c-947c-960e3d1da39b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:09.612717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:09.657806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 656/1322] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:09.702766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48862af0-6ae3-41de-b176-04dc23067016 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:09.704365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-48862af0-6ae3-41de-b176-04dc23067016 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:09.726216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:09.833573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48862af0-6ae3-41de-b176-04dc23067016 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:09.913675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0910b2-193a-4612-b344-86bbbe290579 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:09.914323 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0910b2-193a-4612-b344-86bbbe290579 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:09.914602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0910b2-193a-4612-b344-86bbbe290579 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:09.914874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0910b2-193a-4612-b344-86bbbe290579 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:09.916503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dc0910b2-193a-4612-b344-86bbbe290579 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:09.991720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:10.033930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0910b2-193a-4612-b344-86bbbe290579 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:10.035291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0910b2-193a-4612-b344-86bbbe290579 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:10.055969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 667/1323] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:12.196272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466690d-7b85-4639-a707-d72b95c7f774 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.196825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466690d-7b85-4639-a707-d72b95c7f774 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.196825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1466690d-7b85-4639-a707-d72b95c7f774 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.198247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 657/1324] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:12.203548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b97ee-224a-4245-ac0e-cf17376b9ecc None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.203983 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b97ee-224a-4245-ac0e-cf17376b9ecc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.204227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491b97ee-224a-4245-ac0e-cf17376b9ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.278158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-491b97ee-224a-4245-ac0e-cf17376b9ecc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:12.280391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-491b97ee-224a-4245-ac0e-cf17376b9ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:12.392096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 668/1325] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:12.427691 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466690d-7b85-4639-a707-d72b95c7f774 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:12.429317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1466690d-7b85-4639-a707-d72b95c7f774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:12.558867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1466690d-7b85-4639-a707-d72b95c7f774 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:12.644462 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f0c338-eb6f-408c-a772-6ded7fdb5562 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:12.645114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0c338-eb6f-408c-a772-6ded7fdb5562 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:12.645394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0c338-eb6f-408c-a772-6ded7fdb5562 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:12.645695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f0c338-eb6f-408c-a772-6ded7fdb5562 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:12.647336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c5f0c338-eb6f-408c-a772-6ded7fdb5562 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:12.665293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:12.779695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0c338-eb6f-408c-a772-6ded7fdb5562 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:12.781154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f0c338-eb6f-408c-a772-6ded7fdb5562 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:12.800860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 658/1326] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:14.930005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a456139-4207-4ca7-8657-1db4d60ac3a1 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:14.930587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a456139-4207-4ca7-8657-1db4d60ac3a1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:14.930587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a456139-4207-4ca7-8657-1db4d60ac3a1 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:14.931928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 669/1327] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:14.938467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b920a-5c9b-4207-845b-8968436502a3 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:14.938467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b920a-5c9b-4207-845b-8968436502a3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:14.939042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b920a-5c9b-4207-845b-8968436502a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.010083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-632b920a-5c9b-4207-845b-8968436502a3 None None] MFA Rules not processed for user `343f2a00f3ed43acb00bd614cb69766e`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:15.011297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-632b920a-5c9b-4207-845b-8968436502a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:15.086399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:15.121982 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 659/1328] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:14 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:15.174284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dd9b8-a979-4744-ab55-0aa8832d01e9 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.174573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dd9b8-a979-4744-ab55-0aa8832d01e9 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.174851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db0dd9b8-a979-4744-ab55-0aa8832d01e9 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.176244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 670/1329] 10.222.0.69 () {54 vars in 965 bytes} [Wed Jul 20 19:15:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:15.182199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594055-f80d-45ba-b13e-4abce630837e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.182560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594055-f80d-45ba-b13e-4abce630837e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.182907 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e594055-f80d-45ba-b13e-4abce630837e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.265562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8e594055-f80d-45ba-b13e-4abce630837e None None] MFA Rules not processed for user `3a1a88089d384964bd9774eda4c5d8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:15.266783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8e594055-f80d-45ba-b13e-4abce630837e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:15.348034 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 660/1330] 10.222.0.69 () {58 vars in 1062 bytes} [Wed Jul 20 19:15:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:15:15.357501 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dd9b8-a979-4744-ab55-0aa8832d01e9 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:15.359165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-db0dd9b8-a979-4744-ab55-0aa8832d01e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:15.393922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:15.491263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db0dd9b8-a979-4744-ab55-0aa8832d01e9 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:15.580322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6033dd27-bcea-4ac3-b41d-bcd05d057881 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:15.580975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033dd27-bcea-4ac3-b41d-bcd05d057881 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:15.581259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033dd27-bcea-4ac3-b41d-bcd05d057881 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:15.581571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6033dd27-bcea-4ac3-b41d-bcd05d057881 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:15.583241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6033dd27-bcea-4ac3-b41d-bcd05d057881 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:15.662379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:15.695809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033dd27-bcea-4ac3-b41d-bcd05d057881 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:15.697179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6033dd27-bcea-4ac3-b41d-bcd05d057881 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:15.716886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 671/1331] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:15 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:18.510085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836ae3f-a840-45c8-8d04-dd74a626c3a8 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:18.510689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836ae3f-a840-45c8-8d04-dd74a626c3a8 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:18.510689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1836ae3f-a840-45c8-8d04-dd74a626c3a8 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:18.511950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 661/1332] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:18.517769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2c1bb-b5f8-4582-86ef-2f366a929d8a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:18.518210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2c1bb-b5f8-4582-86ef-2f366a929d8a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:18.518325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ea2c1bb-b5f8-4582-86ef-2f366a929d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:18.584303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8ea2c1bb-b5f8-4582-86ef-2f366a929d8a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:18.585535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8ea2c1bb-b5f8-4582-86ef-2f366a929d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:18.687713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 672/1333] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:18.725360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836ae3f-a840-45c8-8d04-dd74a626c3a8 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:18.727123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1836ae3f-a840-45c8-8d04-dd74a626c3a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:18.806893 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:18.843056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1836ae3f-a840-45c8-8d04-dd74a626c3a8 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:18.919666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63fa2e2b-7811-4239-8b81-54c28beea770 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:18.920388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa2e2b-7811-4239-8b81-54c28beea770 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:18.920669 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa2e2b-7811-4239-8b81-54c28beea770 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:18.920941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63fa2e2b-7811-4239-8b81-54c28beea770 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:18.922777 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-63fa2e2b-7811-4239-8b81-54c28beea770 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:19.036201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa2e2b-7811-4239-8b81-54c28beea770 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:19.037668 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63fa2e2b-7811-4239-8b81-54c28beea770 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:19.047690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:19.056020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 662/1334] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:21.232666 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d3e5f-2ff5-4201-b00e-7f0093a25fc7 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.233265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d3e5f-2ff5-4201-b00e-7f0093a25fc7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.233265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-877d3e5f-2ff5-4201-b00e-7f0093a25fc7 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.234641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 673/1335] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:21.240232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0530-65e1-4662-ab73-082f906ce447 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.240680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0530-65e1-4662-ab73-082f906ce447 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.240791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65f0530-65e1-4662-ab73-082f906ce447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.311933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f65f0530-65e1-4662-ab73-082f906ce447 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:21.313153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f65f0530-65e1-4662-ab73-082f906ce447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:21.415914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 663/1336] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:21.457353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d3e5f-2ff5-4201-b00e-7f0093a25fc7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:21.458964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-877d3e5f-2ff5-4201-b00e-7f0093a25fc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:21.500879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:21.564469 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-877d3e5f-2ff5-4201-b00e-7f0093a25fc7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:21.638955 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342dfa7-097d-4cec-b59c-e495cfa9e959 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:21.639600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342dfa7-097d-4cec-b59c-e495cfa9e959 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:21.639884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342dfa7-097d-4cec-b59c-e495cfa9e959 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:21.640158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342dfa7-097d-4cec-b59c-e495cfa9e959 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:21.641849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f342dfa7-097d-4cec-b59c-e495cfa9e959 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:21.726782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:21.756686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342dfa7-097d-4cec-b59c-e495cfa9e959 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:21.758080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342dfa7-097d-4cec-b59c-e495cfa9e959 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:21.783543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 674/1337] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:24.379595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7823c1-2fcb-4055-9284-edd27bde6729 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:24.380156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7823c1-2fcb-4055-9284-edd27bde6729 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:24.380156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7823c1-2fcb-4055-9284-edd27bde6729 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:24.381557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 664/1338] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:24.386970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fb03b-3fad-49f1-bd62-53a68423803d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:24.387931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fb03b-3fad-49f1-bd62-53a68423803d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:24.387931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570fb03b-3fad-49f1-bd62-53a68423803d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:24.453884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-570fb03b-3fad-49f1-bd62-53a68423803d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:24.454525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-570fb03b-3fad-49f1-bd62-53a68423803d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:24.552098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 675/1339] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:24 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:24.588843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7823c1-2fcb-4055-9284-edd27bde6729 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:24.591075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ee7823c1-2fcb-4055-9284-edd27bde6729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:24.670631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:24.701172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7823c1-2fcb-4055-9284-edd27bde6729 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:24.778766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2f4756-f044-49cf-b130-f0e15aa1d861 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:24.779331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f4756-f044-49cf-b130-f0e15aa1d861 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:24.779609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f4756-f044-49cf-b130-f0e15aa1d861 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:24.779883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2f4756-f044-49cf-b130-f0e15aa1d861 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:24.781718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cc2f4756-f044-49cf-b130-f0e15aa1d861 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:24.894650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f4756-f044-49cf-b130-f0e15aa1d861 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:24.896097 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc2f4756-f044-49cf-b130-f0e15aa1d861 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:24.914739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 665/1340] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:24 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:15:29.090714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05688b0c-8cb3-4f93-a208-32992f31c78a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.091307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05688b0c-8cb3-4f93-a208-32992f31c78a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.091307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05688b0c-8cb3-4f93-a208-32992f31c78a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.092852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 676/1341] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:29.098977 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5691228-6429-4a11-857c-6e8736604651 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.099448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5691228-6429-4a11-857c-6e8736604651 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.099658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5691228-6429-4a11-857c-6e8736604651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.104668 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:29.179053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c5691228-6429-4a11-857c-6e8736604651 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:29.180278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c5691228-6429-4a11-857c-6e8736604651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:29.286873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 666/1342] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:29.328213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05688b0c-8cb3-4f93-a208-32992f31c78a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:29.330146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-05688b0c-8cb3-4f93-a208-32992f31c78a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:29.387889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:29.450276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05688b0c-8cb3-4f93-a208-32992f31c78a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:29.537811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e814a128-97e3-449b-86fd-d9f74a9482b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:29.538378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814a128-97e3-449b-86fd-d9f74a9482b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:29.538679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814a128-97e3-449b-86fd-d9f74a9482b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:29.538931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e814a128-97e3-449b-86fd-d9f74a9482b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:29.540584 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e814a128-97e3-449b-86fd-d9f74a9482b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:29.645443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:29.663926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814a128-97e3-449b-86fd-d9f74a9482b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:29.665365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e814a128-97e3-449b-86fd-d9f74a9482b6 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:29.686958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 677/1343] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:32.390830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0441be0-3277-4643-864f-8a7aaf368649 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.391411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0441be0-3277-4643-864f-8a7aaf368649 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.391411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0441be0-3277-4643-864f-8a7aaf368649 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.392872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 667/1344] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:32.399073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e192d2b-5b2b-4a07-9d65-a2af455f1f07 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.399626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e192d2b-5b2b-4a07-9d65-a2af455f1f07 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.399626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e192d2b-5b2b-4a07-9d65-a2af455f1f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.478245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0e192d2b-5b2b-4a07-9d65-a2af455f1f07 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:32.478763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0e192d2b-5b2b-4a07-9d65-a2af455f1f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:32.592599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 678/1345] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:32 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:15:32.631736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0441be0-3277-4643-864f-8a7aaf368649 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:32.633308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f0441be0-3277-4643-864f-8a7aaf368649 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:32.722239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:32.740530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0441be0-3277-4643-864f-8a7aaf368649 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:32.814720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fef13c-e4c3-4e10-b41d-f7309c6ce2f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:32.815293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fef13c-e4c3-4e10-b41d-f7309c6ce2f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:32.815580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fef13c-e4c3-4e10-b41d-f7309c6ce2f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:32.815861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fef13c-e4c3-4e10-b41d-f7309c6ce2f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:32.817533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-52fef13c-e4c3-4e10-b41d-f7309c6ce2f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:32.928355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fef13c-e4c3-4e10-b41d-f7309c6ce2f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:32.929834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fef13c-e4c3-4e10-b41d-f7309c6ce2f7 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:32.947890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 668/1346] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:32 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:37.114348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05132d53-f9bf-47ba-b982-36acc0e5c9fe None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:37.114952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05132d53-f9bf-47ba-b982-36acc0e5c9fe None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:37.114952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05132d53-f9bf-47ba-b982-36acc0e5c9fe None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:37.116301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 679/1347] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:37.121810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9b82e-ff0b-4b5c-8cb6-3b4c732f6c86 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:37.122277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9b82e-ff0b-4b5c-8cb6-3b4c732f6c86 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:37.122380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f9b82e-ff0b-4b5c-8cb6-3b4c732f6c86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:37.135983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:37.192980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c9f9b82e-ff0b-4b5c-8cb6-3b4c732f6c86 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:37.194284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c9f9b82e-ff0b-4b5c-8cb6-3b4c732f6c86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:37.313300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 669/1348] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:37.351959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05132d53-f9bf-47ba-b982-36acc0e5c9fe None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:37.353535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-05132d53-f9bf-47ba-b982-36acc0e5c9fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:37.415042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:37.465103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05132d53-f9bf-47ba-b982-36acc0e5c9fe None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:37.550533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2da48453-11ee-4afa-9f08-a4fc82275b5b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:37.551195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da48453-11ee-4afa-9f08-a4fc82275b5b service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:37.551481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da48453-11ee-4afa-9f08-a4fc82275b5b service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:37.551760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2da48453-11ee-4afa-9f08-a4fc82275b5b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:37.553436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2da48453-11ee-4afa-9f08-a4fc82275b5b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:37.667291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:37.671155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da48453-11ee-4afa-9f08-a4fc82275b5b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:37.673071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2da48453-11ee-4afa-9f08-a4fc82275b5b service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:37.692260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 680/1349] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:42.709930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef031-98de-40c5-88ab-70d6967fe7be None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:42.710513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef031-98de-40c5-88ab-70d6967fe7be None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:42.710599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-241ef031-98de-40c5-88ab-70d6967fe7be None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:42.711975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 670/1350] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:15:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:42.718966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49da70-1231-47dd-a167-4a12ccbadd8e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:42.719697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49da70-1231-47dd-a167-4a12ccbadd8e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:42.719782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e49da70-1231-47dd-a167-4a12ccbadd8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:42.813082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2e49da70-1231-47dd-a167-4a12ccbadd8e None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:42.814360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2e49da70-1231-47dd-a167-4a12ccbadd8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:42.963180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 681/1351] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:15:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:47.084643 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786df8c9-a38e-4442-a19a-52145f3ef2a7 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.085144 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786df8c9-a38e-4442-a19a-52145f3ef2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.085363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786df8c9-a38e-4442-a19a-52145f3ef2a7 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.087113 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 671/1352] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:15:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:47.092443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62feeacd-ebda-4610-8084-95e3f301b08a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:47.092937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62feeacd-ebda-4610-8084-95e3f301b08a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:47.093032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62feeacd-ebda-4610-8084-95e3f301b08a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:47.150634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:47.175940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-62feeacd-ebda-4610-8084-95e3f301b08a None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:47.177156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-62feeacd-ebda-4610-8084-95e3f301b08a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:47.290930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 682/1353] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:15:47 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:47.311067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=59 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:48.605395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847f113-c47c-40aa-accc-015fcf2b0e25 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:48.606063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847f113-c47c-40aa-accc-015fcf2b0e25 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:48.606063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847f113-c47c-40aa-accc-015fcf2b0e25 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:48.607513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 672/1354] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:15:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:48.612540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a261-ffb3-445f-b4e6-a3e1a95faec3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:48.612980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a261-ffb3-445f-b4e6-a3e1a95faec3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:48.613295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2076a261-ffb3-445f-b4e6-a3e1a95faec3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:48.694287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2076a261-ffb3-445f-b4e6-a3e1a95faec3 None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:48.695486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2076a261-ffb3-445f-b4e6-a3e1a95faec3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:48.738420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:48.808762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 683/1355] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:15:48 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:52.309047 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dbafd-7e10-49fb-86d1-3402d2e4b263 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.309720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dbafd-7e10-49fb-86d1-3402d2e4b263 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.309720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d5dbafd-7e10-49fb-86d1-3402d2e4b263 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.318115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 673/1356] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:52 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:52.328342 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee425974-af7b-4e70-aaca-b6ebc5a0f448 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.328815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee425974-af7b-4e70-aaca-b6ebc5a0f448 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.328918 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee425974-af7b-4e70-aaca-b6ebc5a0f448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.404026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ee425974-af7b-4e70-aaca-b6ebc5a0f448 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:52.405232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ee425974-af7b-4e70-aaca-b6ebc5a0f448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:52.514046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 684/1357] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:52.566313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d09446-b4aa-4e32-adeb-8cd3c98df161 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.566604 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d09446-b4aa-4e32-adeb-8cd3c98df161 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.567127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d09446-b4aa-4e32-adeb-8cd3c98df161 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.568538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 674/1358] 10.222.0.69 () {54 vars in 962 bytes} [Wed Jul 20 19:15:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:52.577330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb8305-b397-46b8-9d5e-9b480cd79592 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:52.577854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb8305-b397-46b8-9d5e-9b480cd79592 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:52.578093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9adb8305-b397-46b8-9d5e-9b480cd79592 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:52.600961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:52.673599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9adb8305-b397-46b8-9d5e-9b480cd79592 None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:52.674796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9adb8305-b397-46b8-9d5e-9b480cd79592 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:52.800543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 685/1359] 10.222.0.69 () {58 vars in 1059 bytes} [Wed Jul 20 19:15:52 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:52.808250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d09446-b4aa-4e32-adeb-8cd3c98df161 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:52.810133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a3d09446-b4aa-4e32-adeb-8cd3c98df161 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:52.992312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d09446-b4aa-4e32-adeb-8cd3c98df161 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:53.100189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:53.143760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5540738-8b27-4349-9d94-dd116070617f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:53.144566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5540738-8b27-4349-9d94-dd116070617f service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:53.144950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5540738-8b27-4349-9d94-dd116070617f service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:53.145347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5540738-8b27-4349-9d94-dd116070617f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:53.147170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f5540738-8b27-4349-9d94-dd116070617f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:53.259407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5540738-8b27-4349-9d94-dd116070617f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:53.261799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5540738-8b27-4349-9d94-dd116070617f service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:53.285741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 675/1360] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 478 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:55.579065 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471c46d-fc90-40f3-8a61-81948dfd8115 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:55.579642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471c46d-fc90-40f3-8a61-81948dfd8115 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:55.579642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471c46d-fc90-40f3-8a61-81948dfd8115 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:55.581157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 686/1361] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:55.586199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714eed8-66e8-4327-8728-f4c35230bfd5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:55.586602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714eed8-66e8-4327-8728-f4c35230bfd5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:55.586770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2714eed8-66e8-4327-8728-f4c35230bfd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:55.661046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2714eed8-66e8-4327-8728-f4c35230bfd5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:55.662727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2714eed8-66e8-4327-8728-f4c35230bfd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:55.705623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:55.785813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 676/1362] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:55.838317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471c46d-fc90-40f3-8a61-81948dfd8115 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:55.840447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2471c46d-fc90-40f3-8a61-81948dfd8115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:55.914213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:56.008952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471c46d-fc90-40f3-8a61-81948dfd8115 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:56.131408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a766320-63cb-4151-be54-9c259e8837bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:56.132054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a766320-63cb-4151-be54-9c259e8837bd service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:56.132338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a766320-63cb-4151-be54-9c259e8837bd service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:56.132609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a766320-63cb-4151-be54-9c259e8837bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:56.134276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7a766320-63cb-4151-be54-9c259e8837bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:56.160262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:56.237171 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a766320-63cb-4151-be54-9c259e8837bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:56.238820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a766320-63cb-4151-be54-9c259e8837bd service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:56.258801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 687/1363] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:15:58.755651 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d043693-7e97-4c24-a79e-d743ed86bf91 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:58.756185 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d043693-7e97-4c24-a79e-d743ed86bf91 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:58.756305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d043693-7e97-4c24-a79e-d743ed86bf91 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:58.757636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 677/1364] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:15:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:58.764397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c24a0-8286-4134-8a08-0c7bbfc9ccca None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:58.764954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c24a0-8286-4134-8a08-0c7bbfc9ccca None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:58.764954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c24a0-8286-4134-8a08-0c7bbfc9ccca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:58.850427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4c8c24a0-8286-4134-8a08-0c7bbfc9ccca None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:58.851595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4c8c24a0-8286-4134-8a08-0c7bbfc9ccca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:58.965436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:58.977867 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 688/1365] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:15:59.018833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dfa30-2dca-44f2-8334-6cebab09b4f6 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:59.019550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dfa30-2dca-44f2-8334-6cebab09b4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:59.019903 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7dfa30-2dca-44f2-8334-6cebab09b4f6 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:59.021382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 678/1366] 10.222.0.69 () {54 vars in 962 bytes} [Wed Jul 20 19:15:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:15:59.027322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859c8ee-a8bf-4a92-9eac-5e1cf08e7a0d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:59.027601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859c8ee-a8bf-4a92-9eac-5e1cf08e7a0d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:59.027883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6859c8ee-a8bf-4a92-9eac-5e1cf08e7a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:59.112358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6859c8ee-a8bf-4a92-9eac-5e1cf08e7a0d None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:15:59.113480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6859c8ee-a8bf-4a92-9eac-5e1cf08e7a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:59.243192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 689/1367] 10.222.0.69 () {58 vars in 1059 bytes} [Wed Jul 20 19:15:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:15:59.250973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7dfa30-2dca-44f2-8334-6cebab09b4f6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:15:59.252472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-dc7dfa30-2dca-44f2-8334-6cebab09b4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:59.413684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7dfa30-2dca-44f2-8334-6cebab09b4f6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:15:59.436523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:15:59.538696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c05ac507-1d5e-40cc-a533-f0a2caa6da7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:15:59.539514 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ac507-1d5e-40cc-a533-f0a2caa6da7a service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:15:59.539916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ac507-1d5e-40cc-a533-f0a2caa6da7a service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:15:59.540292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c05ac507-1d5e-40cc-a533-f0a2caa6da7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:15:59.542420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c05ac507-1d5e-40cc-a533-f0a2caa6da7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:15:59.639904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ac507-1d5e-40cc-a533-f0a2caa6da7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:15:59.642012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c05ac507-1d5e-40cc-a533-f0a2caa6da7a service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:15:59.661688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 679/1368] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:00.741650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e231c72-3612-4f40-a5cb-bee4b6aa3f5b None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:00.742180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e231c72-3612-4f40-a5cb-bee4b6aa3f5b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:00.742292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e231c72-3612-4f40-a5cb-bee4b6aa3f5b None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:00.749910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b970a4b-4c89-440c-8543-2361da223835 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:00.750373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b970a4b-4c89-440c-8543-2361da223835 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:00.750477 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b970a4b-4c89-440c-8543-2361da223835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:00.754069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 690/1369] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:00 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:00.783586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:00.822842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7b970a4b-4c89-440c-8543-2361da223835 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:00.823911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7b970a4b-4c89-440c-8543-2361da223835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:00.938026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 680/1370] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:01.278833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57928a28-b37b-4b0d-bb56-1abc7c20c2ec None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:01.279061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57928a28-b37b-4b0d-bb56-1abc7c20c2ec None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:01.280153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57928a28-b37b-4b0d-bb56-1abc7c20c2ec None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:01.280899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 691/1371] 10.222.0.69 () {54 vars in 964 bytes} [Wed Jul 20 19:16:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:01.285945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa20c1-0f70-49ec-9d24-8945a2036a15 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:01.286308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa20c1-0f70-49ec-9d24-8945a2036a15 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:01.286659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eaa20c1-0f70-49ec-9d24-8945a2036a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:01.382547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0eaa20c1-0f70-49ec-9d24-8945a2036a15 None None] MFA Rules not processed for user `c94d9030f61a4e1f8b3d75ef8925bdd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:01.384346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0eaa20c1-0f70-49ec-9d24-8945a2036a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:01.445030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:01.484104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 681/1372] 10.222.0.69 () {58 vars in 1061 bytes} [Wed Jul 20 19:16:01 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:01.495570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57928a28-b37b-4b0d-bb56-1abc7c20c2ec None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:01.497294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-57928a28-b37b-4b0d-bb56-1abc7c20c2ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:01.565221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:01.638909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57928a28-b37b-4b0d-bb56-1abc7c20c2ec None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:01.745477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb30612-7807-4ca5-a2c7-9a632d421db1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:01.746412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb30612-7807-4ca5-a2c7-9a632d421db1 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:01.746642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb30612-7807-4ca5-a2c7-9a632d421db1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:01.746848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb30612-7807-4ca5-a2c7-9a632d421db1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:01.748668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-deb30612-7807-4ca5-a2c7-9a632d421db1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:01.882595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420bb0d-91e3-44d2-8c49-3ab88c706e4f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:01.883024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420bb0d-91e3-44d2-8c49-3ab88c706e4f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:01.883184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3420bb0d-91e3-44d2-8c49-3ab88c706e4f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:01.883951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:01.889729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 682/1373] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:16:01.891126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c0c99-07ce-42ed-822d-f6b408f32b35 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:01.891419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c0c99-07ce-42ed-822d-f6b408f32b35 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:01.891701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016c0c99-07ce-42ed-822d-f6b408f32b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:01.901394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb30612-7807-4ca5-a2c7-9a632d421db1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:01.902993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb30612-7807-4ca5-a2c7-9a632d421db1 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:01.928738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 692/1374] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:01.990972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-016c0c99-07ce-42ed-822d-f6b408f32b35 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:01.993903 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-016c0c99-07ce-42ed-822d-f6b408f32b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:02.210255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 683/1375] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:02.256269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb30612-7807-4ca5-a2c7-9a632d421db1 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:02.257472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-deb30612-7807-4ca5-a2c7-9a632d421db1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:02.363054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb30612-7807-4ca5-a2c7-9a632d421db1 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:02.468281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:02.481251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afd8eebb-e20a-4f46-a4ae-8b103a422af2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:02.481946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8eebb-e20a-4f46-a4ae-8b103a422af2 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:02.482234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8eebb-e20a-4f46-a4ae-8b103a422af2 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:02.482514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd8eebb-e20a-4f46-a4ae-8b103a422af2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:02.484169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-afd8eebb-e20a-4f46-a4ae-8b103a422af2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:02.577717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8eebb-e20a-4f46-a4ae-8b103a422af2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:02.579274 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afd8eebb-e20a-4f46-a4ae-8b103a422af2 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:02.598728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 693/1376] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:04.708288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4b10a-6100-4035-9d8c-f75f0b7aaa49 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:04.708904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4b10a-6100-4035-9d8c-f75f0b7aaa49 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:04.708904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d4b10a-6100-4035-9d8c-f75f0b7aaa49 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:04.710256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 684/1377] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:04.716333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e792b53-64fe-4d94-ae0a-b95af973aa7a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:04.716799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e792b53-64fe-4d94-ae0a-b95af973aa7a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:04.716908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e792b53-64fe-4d94-ae0a-b95af973aa7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:04.783755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0e792b53-64fe-4d94-ae0a-b95af973aa7a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:04.784968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0e792b53-64fe-4d94-ae0a-b95af973aa7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:04.852360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:04.892622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 694/1378] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:04.938227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4b10a-6100-4035-9d8c-f75f0b7aaa49 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:04.939757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d5d4b10a-6100-4035-9d8c-f75f0b7aaa49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:05.012712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:05.113066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d4b10a-6100-4035-9d8c-f75f0b7aaa49 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:05.225090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c57211f9-09bb-449c-8637-435759e68bbc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:05.225771 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57211f9-09bb-449c-8637-435759e68bbc service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:05.226060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57211f9-09bb-449c-8637-435759e68bbc service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:05.226335 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c57211f9-09bb-449c-8637-435759e68bbc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:05.227975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c57211f9-09bb-449c-8637-435759e68bbc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:05.239708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:05.323602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57211f9-09bb-449c-8637-435759e68bbc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:05.325234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c57211f9-09bb-449c-8637-435759e68bbc service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:05.350675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 685/1379] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 413 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:07.476047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6c9c6-3d4d-4827-b362-c7520009d096 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:07.476617 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6c9c6-3d4d-4827-b362-c7520009d096 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:07.476617 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90a6c9c6-3d4d-4827-b362-c7520009d096 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:07.478102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 695/1380] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:16:07.483970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf7154-f34e-4e76-b956-d06048204561 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:07.484389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf7154-f34e-4e76-b956-d06048204561 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:07.484515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bf7154-f34e-4e76-b956-d06048204561 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:07.555411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-08bf7154-f34e-4e76-b956-d06048204561 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:07.556725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-08bf7154-f34e-4e76-b956-d06048204561 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:07.659433 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:07.673558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 686/1381] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:16:07.718838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6c9c6-3d4d-4827-b362-c7520009d096 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:07.720903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-90a6c9c6-3d4d-4827-b362-c7520009d096 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:07.882020 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90a6c9c6-3d4d-4827-b362-c7520009d096 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:07.937732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:08.007994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1114a364-ca8c-425d-bba2-30103dda6de8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:08.008731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114a364-ca8c-425d-bba2-30103dda6de8 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:08.009078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114a364-ca8c-425d-bba2-30103dda6de8 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:08.009414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1114a364-ca8c-425d-bba2-30103dda6de8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:08.011267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1114a364-ca8c-425d-bba2-30103dda6de8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:08.106961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114a364-ca8c-425d-bba2-30103dda6de8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:08.108418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1114a364-ca8c-425d-bba2-30103dda6de8 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:08.129030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 696/1382] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:10.280296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe4c9b-378d-4710-adcb-7e43a6066421 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:10.281234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe4c9b-378d-4710-adcb-7e43a6066421 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:10.281555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbe4c9b-378d-4710-adcb-7e43a6066421 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:10.283051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 687/1383] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:10.288543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce121c-24ed-4112-96b8-40222408afa3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:10.289004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce121c-24ed-4112-96b8-40222408afa3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:10.289123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ce121c-24ed-4112-96b8-40222408afa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:10.365626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-23ce121c-24ed-4112-96b8-40222408afa3 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:10.366863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-23ce121c-24ed-4112-96b8-40222408afa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:10.368951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:10.470104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 697/1384] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:10.512634 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe4c9b-378d-4710-adcb-7e43a6066421 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:10.514300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-efbe4c9b-378d-4710-adcb-7e43a6066421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:10.673391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbe4c9b-378d-4710-adcb-7e43a6066421 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:10.739644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:10.800850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275c6c8d-4420-4818-991e-5b78d4a465f8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:10.801421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c6c8d-4420-4818-991e-5b78d4a465f8 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:10.801734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c6c8d-4420-4818-991e-5b78d4a465f8 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:10.802012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275c6c8d-4420-4818-991e-5b78d4a465f8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:10.803665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-275c6c8d-4420-4818-991e-5b78d4a465f8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:10.913484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c6c8d-4420-4818-991e-5b78d4a465f8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:10.915599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275c6c8d-4420-4818-991e-5b78d4a465f8 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:10.938579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 688/1385] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:16:13.058877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d0058-644f-4de6-95b3-767efd769e18 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:13.059524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d0058-644f-4de6-95b3-767efd769e18 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:13.059524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020d0058-644f-4de6-95b3-767efd769e18 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:13.060831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 698/1386] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:13.066911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a82e-1218-4587-aaad-373e2cbb2725 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:13.067374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a82e-1218-4587-aaad-373e2cbb2725 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:13.067640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b4a82e-1218-4587-aaad-373e2cbb2725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:13.146883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-46b4a82e-1218-4587-aaad-373e2cbb2725 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:13.148103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-46b4a82e-1218-4587-aaad-373e2cbb2725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:13.175419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:13.249974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 689/1387] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:13.294607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d0058-644f-4de6-95b3-767efd769e18 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:13.296150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-020d0058-644f-4de6-95b3-767efd769e18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:13.440809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:13.459958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020d0058-644f-4de6-95b3-767efd769e18 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:13.588853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25b4c59-b065-470b-a282-5d23771d2595 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:13.589120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b4c59-b065-470b-a282-5d23771d2595 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:13.589541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b4c59-b065-470b-a282-5d23771d2595 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:13.589824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25b4c59-b065-470b-a282-5d23771d2595 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:13.591502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a25b4c59-b065-470b-a282-5d23771d2595 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:13.684928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b4c59-b065-470b-a282-5d23771d2595 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:13.687219 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25b4c59-b065-470b-a282-5d23771d2595 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:13.693904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:13.711220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 699/1388] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:15.805621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6582f-06d4-4976-86e6-9944fdce9903 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:15.806190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6582f-06d4-4976-86e6-9944fdce9903 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:15.806190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b6582f-06d4-4976-86e6-9944fdce9903 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:15.807969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 690/1389] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:15.813154 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f48241-e809-4dee-89aa-4e15e70127ac None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:15.813585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f48241-e809-4dee-89aa-4e15e70127ac None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:15.813751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f48241-e809-4dee-89aa-4e15e70127ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:15.881157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-90f48241-e809-4dee-89aa-4e15e70127ac None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:15.882388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-90f48241-e809-4dee-89aa-4e15e70127ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:15.981168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 700/1390] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:16.025401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6582f-06d4-4976-86e6-9944fdce9903 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:16.027472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-92b6582f-06d4-4976-86e6-9944fdce9903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:16.177142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b6582f-06d4-4976-86e6-9944fdce9903 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:16.186766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:16.300754 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02d8b06a-02b1-49da-a538-13ba26147613 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:16.301399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8b06a-02b1-49da-a538-13ba26147613 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:16.301714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8b06a-02b1-49da-a538-13ba26147613 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:16.301986 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02d8b06a-02b1-49da-a538-13ba26147613 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:16.303622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-02d8b06a-02b1-49da-a538-13ba26147613 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:16.399494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8b06a-02b1-49da-a538-13ba26147613 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:16.401282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02d8b06a-02b1-49da-a538-13ba26147613 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:16.420685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 691/1391] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:17.199583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a6a7c-ddff-4576-ab4e-89f41adc2429 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:17.200158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a6a7c-ddff-4576-ab4e-89f41adc2429 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:17.200158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c0a6a7c-ddff-4576-ab4e-89f41adc2429 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:17.201560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 701/1392] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:16:17.207079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba19bd-aeeb-43db-8648-9e2d2890c6e9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:17.207541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba19bd-aeeb-43db-8648-9e2d2890c6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:17.207639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cba19bd-aeeb-43db-8648-9e2d2890c6e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:17.222613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:17.274043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6cba19bd-aeeb-43db-8648-9e2d2890c6e9 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:17.275251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6cba19bd-aeeb-43db-8648-9e2d2890c6e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:17.376498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 692/1393] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:17.432615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4c40f-d99a-4b43-ac16-e7735598b329 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:17.432988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4c40f-d99a-4b43-ac16-e7735598b329 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:17.433271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b4c40f-d99a-4b43-ac16-e7735598b329 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:17.434811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 702/1394] 10.222.0.69 () {54 vars in 966 bytes} [Wed Jul 20 19:16:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:17.440008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da97de-65d8-42e9-92f4-5599ff880d5a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:17.440295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da97de-65d8-42e9-92f4-5599ff880d5a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:17.440572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19da97de-65d8-42e9-92f4-5599ff880d5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:17.517218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-19da97de-65d8-42e9-92f4-5599ff880d5a None None] MFA Rules not processed for user `3e46ec9c31094d03b6a413e09bc9f699`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:17.518356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-19da97de-65d8-42e9-92f4-5599ff880d5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:17.554618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:17.602218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 693/1395] 10.222.0.69 () {58 vars in 1063 bytes} [Wed Jul 20 19:16:17 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:17.609892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4c40f-d99a-4b43-ac16-e7735598b329 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:17.611430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b6b4c40f-d99a-4b43-ac16-e7735598b329 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:17.686723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:17.719082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b4c40f-d99a-4b43-ac16-e7735598b329 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:17.794558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73170623-1bd0-49d4-ab99-91e2fec7f187 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e46ec9c31094d03b6a413e09bc9f699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:17.795117 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73170623-1bd0-49d4-ab99-91e2fec7f187 service glance] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:17.795396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73170623-1bd0-49d4-ab99-91e2fec7f187 service glance] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:17.795668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73170623-1bd0-49d4-ab99-91e2fec7f187 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:17.797305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-73170623-1bd0-49d4-ab99-91e2fec7f187 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:18.031577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73170623-1bd0-49d4-ab99-91e2fec7f187 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:18.033007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73170623-1bd0-49d4-ab99-91e2fec7f187 service glance] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:18.048574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:18.052287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 703/1396] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:18.518714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bfab4-cdd2-47d3-918b-fd7133f8722a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:18.519289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bfab4-cdd2-47d3-918b-fd7133f8722a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:18.519289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5bfab4-cdd2-47d3-918b-fd7133f8722a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:18.520974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 694/1397] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:18.526577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7f446-2c72-429c-bc1f-d67d9f913487 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:18.526991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7f446-2c72-429c-bc1f-d67d9f913487 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:18.527239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d7f446-2c72-429c-bc1f-d67d9f913487 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:18.594892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-96d7f446-2c72-429c-bc1f-d67d9f913487 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:18.596011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-96d7f446-2c72-429c-bc1f-d67d9f913487 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:18.703234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 704/1398] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:18.749584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bfab4-cdd2-47d3-918b-fd7133f8722a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:18.751455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8c5bfab4-cdd2-47d3-918b-fd7133f8722a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:18.774239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ec6b7-70d8-4972-86e0-383d6b7080eb None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:18.774536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ec6b7-70d8-4972-86e0-383d6b7080eb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:18.774833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9ec6b7-70d8-4972-86e0-383d6b7080eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:18.841957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3c9ec6b7-70d8-4972-86e0-383d6b7080eb None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:18.843171 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3c9ec6b7-70d8-4972-86e0-383d6b7080eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:18.874248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:18.903782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5bfab4-cdd2-47d3-918b-fd7133f8722a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:18.932984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 705/1399] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 19:16:18 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:16:18.945862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:18.981662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bf104-6c26-4b6c-802d-cbebc6149d13 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:18.982034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bf104-6c26-4b6c-802d-cbebc6149d13 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:18.982384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4bf104-6c26-4b6c-802d-cbebc6149d13 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:18.983858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 706/1400] 10.222.0.69 () {54 vars in 943 bytes} [Wed Jul 20 19:16:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:16:18.989772 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57293bb3-ff90-4137-a82d-b9d79f8b3f9b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:18.990175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57293bb3-ff90-4137-a82d-b9d79f8b3f9b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:18.990555 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57293bb3-ff90-4137-a82d-b9d79f8b3f9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:19.028807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6bfc8-623e-4db1-9ffd-7ecba0b2148d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:19.029616 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6bfc8-623e-4db1-9ffd-7ecba0b2148d service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:19.029902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6bfc8-623e-4db1-9ffd-7ecba0b2148d service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:19.030180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba6bfc8-623e-4db1-9ffd-7ecba0b2148d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:19.031851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eba6bfc8-623e-4db1-9ffd-7ecba0b2148d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:19.070477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-57293bb3-ff90-4137-a82d-b9d79f8b3f9b None None] MFA Rules not processed for user `9d8387328e4145db80782f2d54c0966d`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:19.071722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-57293bb3-ff90-4137-a82d-b9d79f8b3f9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:19.123970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6bfc8-623e-4db1-9ffd-7ecba0b2148d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:19.125678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba6bfc8-623e-4db1-9ffd-7ecba0b2148d service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:19.146671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 695/1401] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 398 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:19.187004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 707/1402] 10.222.0.69 () {58 vars in 1040 bytes} [Wed Jul 20 19:16:18 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:16:19.196946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6bfc8-623e-4db1-9ffd-7ecba0b2148d service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:19.198360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eba6bfc8-623e-4db1-9ffd-7ecba0b2148d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:19.253972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:19.301617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba6bfc8-623e-4db1-9ffd-7ecba0b2148d service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:19.417424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245bd2f7-19d2-4100-a9ca-2e0129aff12e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:19.418104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bd2f7-19d2-4100-a9ca-2e0129aff12e service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:19.418384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bd2f7-19d2-4100-a9ca-2e0129aff12e service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:19.418658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245bd2f7-19d2-4100-a9ca-2e0129aff12e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:19.420333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-245bd2f7-19d2-4100-a9ca-2e0129aff12e service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:19.502078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:19.507104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bd2f7-19d2-4100-a9ca-2e0129aff12e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:19.508728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245bd2f7-19d2-4100-a9ca-2e0129aff12e service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:19.517413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 696/1403] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 19:16:19 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:16:21.253813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2849794-c219-468a-91f4-5af5d59b38cf None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:21.254433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2849794-c219-468a-91f4-5af5d59b38cf None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:21.254433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2849794-c219-468a-91f4-5af5d59b38cf None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:21.255921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 708/1404] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:21.262126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5ba49-d89d-4259-9275-fb3da719d59a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:21.262601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5ba49-d89d-4259-9275-fb3da719d59a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:21.262710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c5ba49-d89d-4259-9275-fb3da719d59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:21.352181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-70c5ba49-d89d-4259-9275-fb3da719d59a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:21.354175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-70c5ba49-d89d-4259-9275-fb3da719d59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:21.488000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 697/1405] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:16:21.539289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2849794-c219-468a-91f4-5af5d59b38cf None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:21.541492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a2849794-c219-468a-91f4-5af5d59b38cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:21.545784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:21.728670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2849794-c219-468a-91f4-5af5d59b38cf None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:21.818010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:21.861346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b56309-2b32-4f15-8ee2-f2c35d61e514 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:21.862044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b56309-2b32-4f15-8ee2-f2c35d61e514 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:21.862328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b56309-2b32-4f15-8ee2-f2c35d61e514 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:21.862602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b56309-2b32-4f15-8ee2-f2c35d61e514 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:21.864249 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-12b56309-2b32-4f15-8ee2-f2c35d61e514 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:21.966932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b56309-2b32-4f15-8ee2-f2c35d61e514 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:21.968686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b56309-2b32-4f15-8ee2-f2c35d61e514 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:21.993835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 709/1406] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:24.154409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265a455-3b52-4ef8-b477-20560eda7501 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.154409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265a455-3b52-4ef8-b477-20560eda7501 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.154409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265a455-3b52-4ef8-b477-20560eda7501 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.157879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 698/1407] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:16:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:24.160709 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa00a-377a-4167-bcfa-54691d75ddbc None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.161156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa00a-377a-4167-bcfa-54691d75ddbc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.161307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02aa00a-377a-4167-bcfa-54691d75ddbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.238854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b02aa00a-377a-4167-bcfa-54691d75ddbc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:24.240063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b02aa00a-377a-4167-bcfa-54691d75ddbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:24.283752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:24.360976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 710/1408] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:16:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:24.395570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265a455-3b52-4ef8-b477-20560eda7501 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:24.397333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4265a455-3b52-4ef8-b477-20560eda7501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:24.477170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:24.558345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265a455-3b52-4ef8-b477-20560eda7501 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:24.688443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e1fddea-fd0d-4fb7-848a-362a49da8840 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:24.689107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fddea-fd0d-4fb7-848a-362a49da8840 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:24.689385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fddea-fd0d-4fb7-848a-362a49da8840 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:24.689691 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e1fddea-fd0d-4fb7-848a-362a49da8840 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:24.691395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6e1fddea-fd0d-4fb7-848a-362a49da8840 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:24.735844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:24.809305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fddea-fd0d-4fb7-848a-362a49da8840 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:24.810984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e1fddea-fd0d-4fb7-848a-362a49da8840 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:24.834739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 699/1409] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:34.348305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=26 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:34.832512 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=30 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:42.132239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a19c9-e43d-4656-8fc3-8da351134bab None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:42.132814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a19c9-e43d-4656-8fc3-8da351134bab None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:42.132814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-852a19c9-e43d-4656-8fc3-8da351134bab None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:42.134215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 711/1410] 10.222.0.69 () {54 vars in 918 bytes} [Wed Jul 20 19:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:42.140006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5879-85a1-4473-a54c-5f4e7e33ae42 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:42.140479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5879-85a1-4473-a54c-5f4e7e33ae42 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:42.140588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5879-85a1-4473-a54c-5f4e7e33ae42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:42.215750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5a8a5879-85a1-4473-a54c-5f4e7e33ae42 None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:42.216991 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5a8a5879-85a1-4473-a54c-5f4e7e33ae42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:42.330920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 700/1411] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Jul 20 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:42.345530 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eab0b7-42bd-4fd9-af4d-c93ccc69459e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:42.345827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eab0b7-42bd-4fd9-af4d-c93ccc69459e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:42.346108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eab0b7-42bd-4fd9-af4d-c93ccc69459e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:42.347474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 712/1412] 10.222.0.69 () {54 vars in 918 bytes} [Wed Jul 20 19:16:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:42.352073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111ba87-76d2-4b73-b79c-0721ecbab293 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:42.352383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111ba87-76d2-4b73-b79c-0721ecbab293 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:42.352678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4111ba87-76d2-4b73-b79c-0721ecbab293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:42.424639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:42.428084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4111ba87-76d2-4b73-b79c-0721ecbab293 None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:42.429848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4111ba87-76d2-4b73-b79c-0721ecbab293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:42.546134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 701/1413] 10.222.0.69 () {58 vars in 1015 bytes} [Wed Jul 20 19:16:42 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:16:45.065220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7dc23-6eb1-4b60-a9ce-cf2c444b8af5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:45.065825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7dc23-6eb1-4b60-a9ce-cf2c444b8af5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:45.065825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d7dc23-6eb1-4b60-a9ce-cf2c444b8af5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:45.067232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 713/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:16:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:45.072611 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8791e8-342d-4254-aa36-2b294fb2294e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:45.073659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8791e8-342d-4254-aa36-2b294fb2294e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:45.073800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8791e8-342d-4254-aa36-2b294fb2294e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:45.139174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-eb8791e8-342d-4254-aa36-2b294fb2294e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:45.140383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eb8791e8-342d-4254-aa36-2b294fb2294e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:45.195785 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:45.240289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 702/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:16:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:45.285435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7dc23-6eb1-4b60-a9ce-cf2c444b8af5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:45.287002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-43d7dc23-6eb1-4b60-a9ce-cf2c444b8af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:45.434205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d7dc23-6eb1-4b60-a9ce-cf2c444b8af5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:45.506077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:45.546949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f67822-0f8e-424d-b46c-aa83866eb58c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:45.547601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f67822-0f8e-424d-b46c-aa83866eb58c service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:45.547886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f67822-0f8e-424d-b46c-aa83866eb58c service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:45.548161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f67822-0f8e-424d-b46c-aa83866eb58c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:45.549859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b1f67822-0f8e-424d-b46c-aa83866eb58c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:45.639558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f67822-0f8e-424d-b46c-aa83866eb58c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:45.641067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f67822-0f8e-424d-b46c-aa83866eb58c service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:45.659119 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 714/1416] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:46.564273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de5ee2-69cc-4ef6-ba33-8c73877dded9 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:46.564791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de5ee2-69cc-4ef6-ba33-8c73877dded9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:46.564791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89de5ee2-69cc-4ef6-ba33-8c73877dded9 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:46.566297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 703/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:16:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:46.571115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc626b-443a-4db8-a4f3-3f1e771000f0 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:46.571539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc626b-443a-4db8-a4f3-3f1e771000f0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:46.571676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fc626b-443a-4db8-a4f3-3f1e771000f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:46.649541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a6fc626b-443a-4db8-a4f3-3f1e771000f0 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:46.651381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a6fc626b-443a-4db8-a4f3-3f1e771000f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:46.678472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:46.752130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 715/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:16:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:46.905454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de5ee2-69cc-4ef6-ba33-8c73877dded9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:46.907032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-89de5ee2-69cc-4ef6-ba33-8c73877dded9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:47.054468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89de5ee2-69cc-4ef6-ba33-8c73877dded9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:47.093576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:47.168041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282cd830-3b31-4ead-8a64-a11a73582b74 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:47.168609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cd830-3b31-4ead-8a64-a11a73582b74 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:47.168895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cd830-3b31-4ead-8a64-a11a73582b74 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:47.169173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282cd830-3b31-4ead-8a64-a11a73582b74 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:47.170856 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-282cd830-3b31-4ead-8a64-a11a73582b74 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:47.268001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cd830-3b31-4ead-8a64-a11a73582b74 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:47.269645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282cd830-3b31-4ead-8a64-a11a73582b74 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:47.288012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 704/1419] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 383 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:48.176384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33277e36-ef9f-46c7-b5cb-e8749f110714 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:48.176965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33277e36-ef9f-46c7-b5cb-e8749f110714 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:48.176965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33277e36-ef9f-46c7-b5cb-e8749f110714 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:48.178436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 716/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:16:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:48.183448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec76e3-1f80-4fde-a1b6-1efbfb528875 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:48.183926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec76e3-1f80-4fde-a1b6-1efbfb528875 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:48.184028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ec76e3-1f80-4fde-a1b6-1efbfb528875 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:48.237145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:48.261586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f7ec76e3-1f80-4fde-a1b6-1efbfb528875 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:48.262818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f7ec76e3-1f80-4fde-a1b6-1efbfb528875 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:48.360369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 705/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:16:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:48.511386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33277e36-ef9f-46c7-b5cb-e8749f110714 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:48.512816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-33277e36-ef9f-46c7-b5cb-e8749f110714 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:48.659179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33277e36-ef9f-46c7-b5cb-e8749f110714 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:48.671961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:48.778206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413daf2e-1742-4e24-83ea-dc23797c3f43 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:48.778815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413daf2e-1742-4e24-83ea-dc23797c3f43 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:48.779100 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413daf2e-1742-4e24-83ea-dc23797c3f43 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:48.779379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413daf2e-1742-4e24-83ea-dc23797c3f43 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:48.781053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-413daf2e-1742-4e24-83ea-dc23797c3f43 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:48.870783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413daf2e-1742-4e24-83ea-dc23797c3f43 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:48.872376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413daf2e-1742-4e24-83ea-dc23797c3f43 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:48.890514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 717/1422] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:49.756684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ccd80-acff-458a-a91b-b0d4551b6a02 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:49.757253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ccd80-acff-458a-a91b-b0d4551b6a02 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:49.757253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154ccd80-acff-458a-a91b-b0d4551b6a02 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:49.758714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 706/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:16:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:49.764577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92241503-72ec-413c-a809-9b75467eee5f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:49.765061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92241503-72ec-413c-a809-9b75467eee5f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:49.765245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92241503-72ec-413c-a809-9b75467eee5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:49.783849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:49.843597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-92241503-72ec-413c-a809-9b75467eee5f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:49.845101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-92241503-72ec-413c-a809-9b75467eee5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:49.947427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 718/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:16:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:50.102900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ccd80-acff-458a-a91b-b0d4551b6a02 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:50.104399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-154ccd80-acff-458a-a91b-b0d4551b6a02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:50.221842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:50.262937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154ccd80-acff-458a-a91b-b0d4551b6a02 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:50.402075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5840ef66-6279-4ac5-841c-48ab812ed29e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:50.402743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840ef66-6279-4ac5-841c-48ab812ed29e service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:50.403030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840ef66-6279-4ac5-841c-48ab812ed29e service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:50.403308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5840ef66-6279-4ac5-841c-48ab812ed29e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:50.405012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5840ef66-6279-4ac5-841c-48ab812ed29e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:50.484240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:50.498488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840ef66-6279-4ac5-841c-48ab812ed29e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:50.500127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5840ef66-6279-4ac5-841c-48ab812ed29e service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:50.518709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 707/1425] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 416 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:51.393200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0642539-11c9-4375-8dc6-d5bf5b7d72bd None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:51.393773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0642539-11c9-4375-8dc6-d5bf5b7d72bd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:51.393773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0642539-11c9-4375-8dc6-d5bf5b7d72bd None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:51.395165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 719/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:16:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:51.400040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28ff9a-1e5e-4de9-a51c-f4209bc247c6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:51.400464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28ff9a-1e5e-4de9-a51c-f4209bc247c6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:51.400598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d28ff9a-1e5e-4de9-a51c-f4209bc247c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:51.474779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7d28ff9a-1e5e-4de9-a51c-f4209bc247c6 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:51.475966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7d28ff9a-1e5e-4de9-a51c-f4209bc247c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:51.571862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 708/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:16:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:51.726213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0642539-11c9-4375-8dc6-d5bf5b7d72bd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:51.727757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f0642539-11c9-4375-8dc6-d5bf5b7d72bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:51.810351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:51.872024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0642539-11c9-4375-8dc6-d5bf5b7d72bd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:51.983358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b829ca-e475-41fe-bab1-517c29a12b33 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:51.984005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b829ca-e475-41fe-bab1-517c29a12b33 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:51.984289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b829ca-e475-41fe-bab1-517c29a12b33 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:51.984572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b829ca-e475-41fe-bab1-517c29a12b33 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:51.986247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-37b829ca-e475-41fe-bab1-517c29a12b33 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:52.035252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:52.077444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b829ca-e475-41fe-bab1-517c29a12b33 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:52.079064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b829ca-e475-41fe-bab1-517c29a12b33 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:52.097285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 720/1428] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 372 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:52.907288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35438f8c-5611-4c4c-956a-f2513cdd5251 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:52.907989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35438f8c-5611-4c4c-956a-f2513cdd5251 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:52.907989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35438f8c-5611-4c4c-956a-f2513cdd5251 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:52.909393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 709/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:16:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:52.914810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0927d8e-9c19-4ae3-9a5a-f60cd9149271 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:52.915311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0927d8e-9c19-4ae3-9a5a-f60cd9149271 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:52.915416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0927d8e-9c19-4ae3-9a5a-f60cd9149271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:53.001125 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d0927d8e-9c19-4ae3-9a5a-f60cd9149271 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:53.002554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d0927d8e-9c19-4ae3-9a5a-f60cd9149271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:53.098577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 721/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:16:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:53.252557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35438f8c-5611-4c4c-956a-f2513cdd5251 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:53.254241 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-35438f8c-5611-4c4c-956a-f2513cdd5251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:53.305356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:53.407976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35438f8c-5611-4c4c-956a-f2513cdd5251 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:53.519778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943d95c3-150f-4747-96db-0468b24aa217 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:53.520346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d95c3-150f-4747-96db-0468b24aa217 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:53.520632 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d95c3-150f-4747-96db-0468b24aa217 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:53.520908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943d95c3-150f-4747-96db-0468b24aa217 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:53.522600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-943d95c3-150f-4747-96db-0468b24aa217 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:53.529569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:53.612734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d95c3-150f-4747-96db-0468b24aa217 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:53.614393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943d95c3-150f-4747-96db-0468b24aa217 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:53.632760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 710/1431] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:54.438463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64caf4-b71f-4473-8301-374bca873704 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:54.439006 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64caf4-b71f-4473-8301-374bca873704 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:54.439006 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a64caf4-b71f-4473-8301-374bca873704 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:54.440334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 722/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:16:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:54.445215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34221d-8645-4c95-91d3-fd5796820e46 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:54.445686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34221d-8645-4c95-91d3-fd5796820e46 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:54.445823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f34221d-8645-4c95-91d3-fd5796820e46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:54.524127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8f34221d-8645-4c95-91d3-fd5796820e46 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:54.525357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8f34221d-8645-4c95-91d3-fd5796820e46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:54.596431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:54.625883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 711/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:54.778001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64caf4-b71f-4473-8301-374bca873704 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:54.779486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5a64caf4-b71f-4473-8301-374bca873704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:54.798711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:54.935623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a64caf4-b71f-4473-8301-374bca873704 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:55.034091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:55.061895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a118ac-415d-46f8-9694-7dc1aa80b01e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:55.062548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a118ac-415d-46f8-9694-7dc1aa80b01e service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:55.062830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a118ac-415d-46f8-9694-7dc1aa80b01e service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:55.063108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a118ac-415d-46f8-9694-7dc1aa80b01e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:55.064777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-04a118ac-415d-46f8-9694-7dc1aa80b01e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:55.167303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a118ac-415d-46f8-9694-7dc1aa80b01e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:55.168915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a118ac-415d-46f8-9694-7dc1aa80b01e service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:55.189203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 723/1434] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:56.014023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847af59-cc40-43ab-a0d8-2facd2089d31 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:56.014581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847af59-cc40-43ab-a0d8-2facd2089d31 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:56.014581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2847af59-cc40-43ab-a0d8-2facd2089d31 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:56.015965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 712/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:16:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:56.020976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26527db8-8543-4658-8ea5-8e26f236c6d1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:56.021439 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26527db8-8543-4658-8ea5-8e26f236c6d1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:56.021581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26527db8-8543-4658-8ea5-8e26f236c6d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:56.098439 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-26527db8-8543-4658-8ea5-8e26f236c6d1 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:56.099570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-26527db8-8543-4658-8ea5-8e26f236c6d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:56.140902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:56.197054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 724/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:16:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:56.347612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847af59-cc40-43ab-a0d8-2facd2089d31 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:56.349170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2847af59-cc40-43ab-a0d8-2facd2089d31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:56.492033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2847af59-cc40-43ab-a0d8-2facd2089d31 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:56.547245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:56.606291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665cab2-14fb-4541-b4c6-bd606c0233d0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:56.606927 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665cab2-14fb-4541-b4c6-bd606c0233d0 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:56.607214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665cab2-14fb-4541-b4c6-bd606c0233d0 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:56.607492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665cab2-14fb-4541-b4c6-bd606c0233d0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:56.609151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4665cab2-14fb-4541-b4c6-bd606c0233d0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:56.700354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665cab2-14fb-4541-b4c6-bd606c0233d0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:56.701989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4665cab2-14fb-4541-b4c6-bd606c0233d0 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:56.721043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 713/1437] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:16:57.535129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60aa86-dfb0-4ec0-a81c-8fb480b8eaeb None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:57.535712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60aa86-dfb0-4ec0-a81c-8fb480b8eaeb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:57.535712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a60aa86-dfb0-4ec0-a81c-8fb480b8eaeb None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:57.537086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 725/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:16:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:16:57.542725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9cf1-b8ef-4bdf-b15e-d6526ce42881 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:57.543117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9cf1-b8ef-4bdf-b15e-d6526ce42881 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:57.543287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664c9cf1-b8ef-4bdf-b15e-d6526ce42881 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:57.622088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-664c9cf1-b8ef-4bdf-b15e-d6526ce42881 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:16:57.623222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-664c9cf1-b8ef-4bdf-b15e-d6526ce42881 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:57.625312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:57.736105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 714/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:16:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:16:57.891620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60aa86-dfb0-4ec0-a81c-8fb480b8eaeb None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:16:57.893079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2a60aa86-dfb0-4ec0-a81c-8fb480b8eaeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:58.049241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a60aa86-dfb0-4ec0-a81c-8fb480b8eaeb None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:16:58.080196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:16:58.178981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3415efd-8ab4-44d0-953f-eda726764161 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:16:58.179608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3415efd-8ab4-44d0-953f-eda726764161 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:16:58.179893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3415efd-8ab4-44d0-953f-eda726764161 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:16:58.180161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3415efd-8ab4-44d0-953f-eda726764161 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:16:58.182358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e3415efd-8ab4-44d0-953f-eda726764161 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:16:58.285221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3415efd-8ab4-44d0-953f-eda726764161 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:16:58.286558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3415efd-8ab4-44d0-953f-eda726764161 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:16:58.308590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 726/1440] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:16:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 417 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:00.308910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e74ef-ad15-4fe7-a5bd-c11e0acc5c4b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:00.309533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e74ef-ad15-4fe7-a5bd-c11e0acc5c4b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:00.309533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5e74ef-ad15-4fe7-a5bd-c11e0acc5c4b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:00.310854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 715/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:00.316142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680e217-9a4f-4bb2-90fc-ee34da6d9eea None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:00.316577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680e217-9a4f-4bb2-90fc-ee34da6d9eea None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:00.316686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e680e217-9a4f-4bb2-90fc-ee34da6d9eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:00.360267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:00.387458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e680e217-9a4f-4bb2-90fc-ee34da6d9eea None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:00.388892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e680e217-9a4f-4bb2-90fc-ee34da6d9eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:00.483840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 727/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:00.523327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e74ef-ad15-4fe7-a5bd-c11e0acc5c4b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:00.524866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8c5e74ef-ad15-4fe7-a5bd-c11e0acc5c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:00.629016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5e74ef-ad15-4fe7-a5bd-c11e0acc5c4b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:00.660714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:00.702202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab1fa59-53d0-42a8-8c5f-28568b4d161f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:00.702847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1fa59-53d0-42a8-8c5f-28568b4d161f service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:00.703124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1fa59-53d0-42a8-8c5f-28568b4d161f service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:00.703392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab1fa59-53d0-42a8-8c5f-28568b4d161f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:00.705015 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5ab1fa59-53d0-42a8-8c5f-28568b4d161f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:00.814950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1fa59-53d0-42a8-8c5f-28568b4d161f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:00.816317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab1fa59-53d0-42a8-8c5f-28568b4d161f service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:00.834508 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 716/1443] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:03.138919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d0693-13b6-4964-8f19-9835947e5b24 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:03.139463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d0693-13b6-4964-8f19-9835947e5b24 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:03.139463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f3d0693-13b6-4964-8f19-9835947e5b24 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:03.140766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 728/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:03.146135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339aeeb-1cda-43c8-bab6-6a14f8ebcc46 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:03.146538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339aeeb-1cda-43c8-bab6-6a14f8ebcc46 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:03.146707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c339aeeb-1cda-43c8-bab6-6a14f8ebcc46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:03.210467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c339aeeb-1cda-43c8-bab6-6a14f8ebcc46 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:03.211677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c339aeeb-1cda-43c8-bab6-6a14f8ebcc46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:03.225928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:03.309938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 717/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:03.349250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d0693-13b6-4964-8f19-9835947e5b24 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:03.350840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7f3d0693-13b6-4964-8f19-9835947e5b24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:03.454975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f3d0693-13b6-4964-8f19-9835947e5b24 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:03.466963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:03.529764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafea09f-ed8b-4864-b1b7-3750f2cbf9fa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:03.530419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea09f-ed8b-4864-b1b7-3750f2cbf9fa service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:03.530705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea09f-ed8b-4864-b1b7-3750f2cbf9fa service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:03.530989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafea09f-ed8b-4864-b1b7-3750f2cbf9fa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:03.532662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dafea09f-ed8b-4864-b1b7-3750f2cbf9fa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:03.643035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafea09f-ed8b-4864-b1b7-3750f2cbf9fa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:03.644490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafea09f-ed8b-4864-b1b7-3750f2cbf9fa service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:03.662977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 729/1446] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:06.015997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2aa3e-9fee-4a1e-87e6-79d26605a1d7 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:06.016596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2aa3e-9fee-4a1e-87e6-79d26605a1d7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:06.016596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db2aa3e-9fee-4a1e-87e6-79d26605a1d7 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:06.017980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 718/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:06.023680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d94ec3-a69a-4876-9959-3a4de11bb5ca None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:06.024109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d94ec3-a69a-4876-9959-3a4de11bb5ca None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:06.024247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d94ec3-a69a-4876-9959-3a4de11bb5ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:06.071113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:06.090378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-93d94ec3-a69a-4876-9959-3a4de11bb5ca None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:06.091600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-93d94ec3-a69a-4876-9959-3a4de11bb5ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:06.188935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 730/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:06.229231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2aa3e-9fee-4a1e-87e6-79d26605a1d7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:06.230877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1db2aa3e-9fee-4a1e-87e6-79d26605a1d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:06.337485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db2aa3e-9fee-4a1e-87e6-79d26605a1d7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:06.377615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:06.412689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1559562-89d9-4478-a151-726fbb70de17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:06.413338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559562-89d9-4478-a151-726fbb70de17 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:06.413664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559562-89d9-4478-a151-726fbb70de17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:06.413943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1559562-89d9-4478-a151-726fbb70de17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:06.415587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e1559562-89d9-4478-a151-726fbb70de17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:06.525563 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1559562-89d9-4478-a151-726fbb70de17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:06.527138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1559562-89d9-4478-a151-726fbb70de17 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:06.545380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 719/1449] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:08.712397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018da0e-94f7-45b6-a1b0-41ddf7c36398 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:08.713454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018da0e-94f7-45b6-a1b0-41ddf7c36398 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:08.713573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c018da0e-94f7-45b6-a1b0-41ddf7c36398 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:08.715135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 731/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:08.720675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c155225-2cf3-4a98-889a-56599e73cd32 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:08.721110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c155225-2cf3-4a98-889a-56599e73cd32 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:08.721223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c155225-2cf3-4a98-889a-56599e73cd32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:08.787105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2c155225-2cf3-4a98-889a-56599e73cd32 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:08.788310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2c155225-2cf3-4a98-889a-56599e73cd32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:08.809362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:08.886126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 720/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:08.925778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018da0e-94f7-45b6-a1b0-41ddf7c36398 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:08.927307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c018da0e-94f7-45b6-a1b0-41ddf7c36398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:09.032481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c018da0e-94f7-45b6-a1b0-41ddf7c36398 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:09.052558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:09.108179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5414cebe-5017-43f1-b610-926b232d78cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:09.108824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414cebe-5017-43f1-b610-926b232d78cf service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:09.109104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414cebe-5017-43f1-b610-926b232d78cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:09.109378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5414cebe-5017-43f1-b610-926b232d78cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:09.111043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5414cebe-5017-43f1-b610-926b232d78cf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:09.220427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414cebe-5017-43f1-b610-926b232d78cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:09.221899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5414cebe-5017-43f1-b610-926b232d78cf service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:09.240110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 732/1452] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:11.458811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27453a4-cf53-4fd4-81ae-b75313c8b4f4 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:11.459376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27453a4-cf53-4fd4-81ae-b75313c8b4f4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:11.459493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27453a4-cf53-4fd4-81ae-b75313c8b4f4 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:11.460780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 721/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:11.466996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ec40c-e5c0-43b7-b32e-057a2b77a987 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:11.467447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ec40c-e5c0-43b7-b32e-057a2b77a987 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:11.467555 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4ec40c-e5c0-43b7-b32e-057a2b77a987 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:11.532524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:11.536071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4e4ec40c-e5c0-43b7-b32e-057a2b77a987 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:11.537451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4e4ec40c-e5c0-43b7-b32e-057a2b77a987 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:11.637205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 733/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:11.675801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27453a4-cf53-4fd4-81ae-b75313c8b4f4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:11.677352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b27453a4-cf53-4fd4-81ae-b75313c8b4f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:11.781536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27453a4-cf53-4fd4-81ae-b75313c8b4f4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:11.831114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:11.857301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231182b4-7e98-445b-a892-e8be38528019 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:11.857987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231182b4-7e98-445b-a892-e8be38528019 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:11.858276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231182b4-7e98-445b-a892-e8be38528019 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:11.858558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231182b4-7e98-445b-a892-e8be38528019 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:11.860230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-231182b4-7e98-445b-a892-e8be38528019 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:11.968542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231182b4-7e98-445b-a892-e8be38528019 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:11.970284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231182b4-7e98-445b-a892-e8be38528019 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:11.989030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 722/1455] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:14.129648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddb4f8-24ef-446c-86f7-c311f0a8dab4 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:14.130193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddb4f8-24ef-446c-86f7-c311f0a8dab4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:14.130193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ddb4f8-24ef-446c-86f7-c311f0a8dab4 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:14.131601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 734/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:14.137753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce612de-933b-4064-87fc-a00eb96bf385 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:14.138174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce612de-933b-4064-87fc-a00eb96bf385 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:14.138304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce612de-933b-4064-87fc-a00eb96bf385 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:14.204654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7ce612de-933b-4064-87fc-a00eb96bf385 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:14.205927 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7ce612de-933b-4064-87fc-a00eb96bf385 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:14.235611 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:14.307018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 723/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:14.345940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddb4f8-24ef-446c-86f7-c311f0a8dab4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:14.347487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-31ddb4f8-24ef-446c-86f7-c311f0a8dab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:14.453048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ddb4f8-24ef-446c-86f7-c311f0a8dab4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:14.481738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:14.527451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec735692-24a8-4967-8b3d-feea20674e21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:14.528096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735692-24a8-4967-8b3d-feea20674e21 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:14.528375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735692-24a8-4967-8b3d-feea20674e21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:14.528646 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec735692-24a8-4967-8b3d-feea20674e21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:14.530320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ec735692-24a8-4967-8b3d-feea20674e21 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:14.638755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec735692-24a8-4967-8b3d-feea20674e21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:14.640189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec735692-24a8-4967-8b3d-feea20674e21 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:14.658989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 735/1458] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:16.787485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611c824-7010-4e32-b5f1-5e96e808a731 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:16.788012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611c824-7010-4e32-b5f1-5e96e808a731 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:16.788012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611c824-7010-4e32-b5f1-5e96e808a731 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:16.789425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 724/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:16.795179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e21e1-52c0-4ef6-9240-4ac36dbf4adc None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:16.795577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e21e1-52c0-4ef6-9240-4ac36dbf4adc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:16.795774 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371e21e1-52c0-4ef6-9240-4ac36dbf4adc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:16.864277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-371e21e1-52c0-4ef6-9240-4ac36dbf4adc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:16.865528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-371e21e1-52c0-4ef6-9240-4ac36dbf4adc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:16.874096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:16.971631 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 736/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:17.010905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611c824-7010-4e32-b5f1-5e96e808a731 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:17.012466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b611c824-7010-4e32-b5f1-5e96e808a731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:17.123532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611c824-7010-4e32-b5f1-5e96e808a731 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:17.179759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:17.199837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c4695f2-658c-4770-95e1-ab040f84fa1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:17.200492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4695f2-658c-4770-95e1-ab040f84fa1a service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:17.200781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4695f2-658c-4770-95e1-ab040f84fa1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:17.201057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c4695f2-658c-4770-95e1-ab040f84fa1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:17.202779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5c4695f2-658c-4770-95e1-ab040f84fa1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:17.313501 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4695f2-658c-4770-95e1-ab040f84fa1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:17.314921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c4695f2-658c-4770-95e1-ab040f84fa1a service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:17.333067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 725/1461] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:19.428304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee5a85-d707-4436-858d-46df38c4cfa3 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:19.428937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee5a85-d707-4436-858d-46df38c4cfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:19.428937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ee5a85-d707-4436-858d-46df38c4cfa3 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:19.430391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 737/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:19.436763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cb2cf-981d-4534-8836-7dc310a6d0fa None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:19.437173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cb2cf-981d-4534-8836-7dc310a6d0fa None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:19.437308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1cb2cf-981d-4534-8836-7dc310a6d0fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:19.513537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1e1cb2cf-981d-4534-8836-7dc310a6d0fa None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:19.514783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1e1cb2cf-981d-4534-8836-7dc310a6d0fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:19.555749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:19.620625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 726/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:19.657883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee5a85-d707-4436-858d-46df38c4cfa3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:19.659553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-23ee5a85-d707-4436-858d-46df38c4cfa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:19.765342 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ee5a85-d707-4436-858d-46df38c4cfa3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:19.802679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:19.840137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de80448-55d9-4f48-b3eb-4b648ba82e9a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:19.840783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de80448-55d9-4f48-b3eb-4b648ba82e9a service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:19.841067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de80448-55d9-4f48-b3eb-4b648ba82e9a service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:19.841347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de80448-55d9-4f48-b3eb-4b648ba82e9a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:19.843159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3de80448-55d9-4f48-b3eb-4b648ba82e9a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:19.952057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de80448-55d9-4f48-b3eb-4b648ba82e9a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:19.953530 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de80448-55d9-4f48-b3eb-4b648ba82e9a service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:19.972821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 738/1464] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:22.013247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d366c-6043-4bd3-8f71-437c8639a6d4 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:22.013813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d366c-6043-4bd3-8f71-437c8639a6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:22.013945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-761d366c-6043-4bd3-8f71-437c8639a6d4 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:22.015245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 727/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:22.020577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c53047-cec8-4078-9516-b8491f50ae1c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:22.021023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c53047-cec8-4078-9516-b8491f50ae1c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:22.021121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c53047-cec8-4078-9516-b8491f50ae1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:22.085290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-19c53047-cec8-4078-9516-b8491f50ae1c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:22.086536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-19c53047-cec8-4078-9516-b8491f50ae1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:22.105540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:22.185499 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 739/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:22.223032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d366c-6043-4bd3-8f71-437c8639a6d4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:22.224562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-761d366c-6043-4bd3-8f71-437c8639a6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:22.330645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-761d366c-6043-4bd3-8f71-437c8639a6d4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:22.398505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:22.406098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1fe28b-6f46-4ba9-b0be-0e92f02b0b2a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:22.406757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe28b-6f46-4ba9-b0be-0e92f02b0b2a service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:22.407039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe28b-6f46-4ba9-b0be-0e92f02b0b2a service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:22.407312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1fe28b-6f46-4ba9-b0be-0e92f02b0b2a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:22.408982 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0c1fe28b-6f46-4ba9-b0be-0e92f02b0b2a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:22.517874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fe28b-6f46-4ba9-b0be-0e92f02b0b2a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:22.519245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1fe28b-6f46-4ba9-b0be-0e92f02b0b2a service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:22.537680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 728/1467] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:24.596708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ac532-4de8-4622-b607-390e64332970 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:24.597297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ac532-4de8-4622-b607-390e64332970 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:24.597404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1ac532-4de8-4622-b607-390e64332970 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:24.598799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 740/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:17:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:24.604716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0d51a-7723-43c4-b845-be32224f4283 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:24.605179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0d51a-7723-43c4-b845-be32224f4283 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:24.605303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cc0d51a-7723-43c4-b845-be32224f4283 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:24.670365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5cc0d51a-7723-43c4-b845-be32224f4283 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:24.671578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5cc0d51a-7723-43c4-b845-be32224f4283 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:24.722952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:24.775412 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 729/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:17:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:17:24.814307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ac532-4de8-4622-b607-390e64332970 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:24.816057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fc1ac532-4de8-4622-b607-390e64332970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:24.933904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1ac532-4de8-4622-b607-390e64332970 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:24.984413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:25.011976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9561b8ef-7854-4325-a2f2-14f7a334c3fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:25.012914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561b8ef-7854-4325-a2f2-14f7a334c3fb service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:25.013204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561b8ef-7854-4325-a2f2-14f7a334c3fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:25.013478 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9561b8ef-7854-4325-a2f2-14f7a334c3fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:25.015165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9561b8ef-7854-4325-a2f2-14f7a334c3fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:25.126123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561b8ef-7854-4325-a2f2-14f7a334c3fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:25.129693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9561b8ef-7854-4325-a2f2-14f7a334c3fb service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:25.149707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 741/1470] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:34.773353 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=22 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:35.147411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=66 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:45.154659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f8e34-64f8-41b0-96a6-171ee876bee3 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:45.155207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f8e34-64f8-41b0-96a6-171ee876bee3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:45.155328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2f8e34-64f8-41b0-96a6-171ee876bee3 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:45.156821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 730/1471] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:45.162484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6378677-8e66-49c4-8f0c-09a293db2c49 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:45.162892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6378677-8e66-49c4-8f0c-09a293db2c49 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:45.163031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6378677-8e66-49c4-8f0c-09a293db2c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:45.227654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c6378677-8e66-49c4-8f0c-09a293db2c49 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:45.228856 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c6378677-8e66-49c4-8f0c-09a293db2c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:45.325882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 742/1472] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:45.411697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97933f5e-70ba-435e-8a4b-f3aa75621fad None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:45.411983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97933f5e-70ba-435e-8a4b-f3aa75621fad None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:45.412260 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97933f5e-70ba-435e-8a4b-f3aa75621fad None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:45.413657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 731/1473] 10.222.0.69 () {54 vars in 966 bytes} [Wed Jul 20 19:17:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:45.418666 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fa3dc-5902-467c-969d-9774bfa4b6be None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:45.418959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fa3dc-5902-467c-969d-9774bfa4b6be None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:45.419240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9fa3dc-5902-467c-969d-9774bfa4b6be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:45.497152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-fd9fa3dc-5902-467c-969d-9774bfa4b6be None None] MFA Rules not processed for user `3e46ec9c31094d03b6a413e09bc9f699`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:45.498432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fd9fa3dc-5902-467c-969d-9774bfa4b6be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:45.519682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:45.576334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 743/1474] 10.222.0.69 () {58 vars in 1063 bytes} [Wed Jul 20 19:17:45 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:45.583905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97933f5e-70ba-435e-8a4b-f3aa75621fad None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:45.585428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-97933f5e-70ba-435e-8a4b-f3aa75621fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:45.690502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97933f5e-70ba-435e-8a4b-f3aa75621fad None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:45.768316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e13dfbe-3f23-4fa6-bbe9-eb56cbfb8fce service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e46ec9c31094d03b6a413e09bc9f699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:45.768973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13dfbe-3f23-4fa6-bbe9-eb56cbfb8fce service glance] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:45.769259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13dfbe-3f23-4fa6-bbe9-eb56cbfb8fce service glance] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:45.769562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e13dfbe-3f23-4fa6-bbe9-eb56cbfb8fce service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:45.771231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9e13dfbe-3f23-4fa6-bbe9-eb56cbfb8fce service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:45.826782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:45.882245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13dfbe-3f23-4fa6-bbe9-eb56cbfb8fce service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:45.884002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e13dfbe-3f23-4fa6-bbe9-eb56cbfb8fce service glance] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:45.904225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 732/1475] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:48.038907 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32f46e-f3a9-4c40-9221-92a4c352304e None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:48.039562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32f46e-f3a9-4c40-9221-92a4c352304e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:48.039562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f32f46e-f3a9-4c40-9221-92a4c352304e None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:48.041528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 744/1476] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:17:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:48.046938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0bdb7-e4bf-4712-9b28-9341b1f10b76 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:48.047365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0bdb7-e4bf-4712-9b28-9341b1f10b76 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:48.047500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c0bdb7-e4bf-4712-9b28-9341b1f10b76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:48.114361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c5c0bdb7-e4bf-4712-9b28-9341b1f10b76 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:48.115576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c5c0bdb7-e4bf-4712-9b28-9341b1f10b76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:48.215193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 733/1477] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:48.239579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32f46e-f3a9-4c40-9221-92a4c352304e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:48.241124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3f32f46e-f3a9-4c40-9221-92a4c352304e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:48.368849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f32f46e-f3a9-4c40-9221-92a4c352304e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:48.419100 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:48.467169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24a0f60-1696-442d-b68f-bd150ffe9ef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:48.467828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a0f60-1696-442d-b68f-bd150ffe9ef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:48.468157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a0f60-1696-442d-b68f-bd150ffe9ef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:48.468373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24a0f60-1696-442d-b68f-bd150ffe9ef7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:48.469105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a0f60-1696-442d-b68f-bd150ffe9ef7 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:48.471105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-b24a0f60-1696-442d-b68f-bd150ffe9ef7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:17:48.471105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 20 19:17:48.471105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: Jul 20 19:17:48.472239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24a0f60-1696-442d-b68f-bd150ffe9ef7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:48.491583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 745/1478] 10.222.0.69 () {56 vars in 1108 bytes} [Wed Jul 20 19:17:48 2022] GET /identity/v3/projects => generated 2553 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:17:50.591056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96d5de-24d7-4479-b8d6-178199197d08 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:50.591581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96d5de-24d7-4479-b8d6-178199197d08 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:50.591581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca96d5de-24d7-4479-b8d6-178199197d08 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:50.592915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 734/1479] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:17:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:50.598615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ed0d0-2167-4946-86f6-436b45c4ccd4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:50.599074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ed0d0-2167-4946-86f6-436b45c4ccd4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:50.599191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7ed0d0-2167-4946-86f6-436b45c4ccd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:50.665598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6d7ed0d0-2167-4946-86f6-436b45c4ccd4 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:50.666873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6d7ed0d0-2167-4946-86f6-436b45c4ccd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:50.764546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 746/1480] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:50.808959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96d5de-24d7-4479-b8d6-178199197d08 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:50.810542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ca96d5de-24d7-4479-b8d6-178199197d08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:50.820196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:50.953902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca96d5de-24d7-4479-b8d6-178199197d08 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:51.035270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:51.069338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f66c71-91cd-4f21-a95a-b7ee291d744c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:51.070028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f66c71-91cd-4f21-a95a-b7ee291d744c service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:51.070318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f66c71-91cd-4f21-a95a-b7ee291d744c service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:51.071106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f66c71-91cd-4f21-a95a-b7ee291d744c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:51.072806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-78f66c71-91cd-4f21-a95a-b7ee291d744c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:51.163771 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f66c71-91cd-4f21-a95a-b7ee291d744c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:51.165604 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f66c71-91cd-4f21-a95a-b7ee291d744c service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:51.183720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 735/1481] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:53.412752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d05a2-5a48-4c49-80a3-41ed16e812ec None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:53.413321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d05a2-5a48-4c49-80a3-41ed16e812ec None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:53.413321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e2d05a2-5a48-4c49-80a3-41ed16e812ec None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:53.415334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 747/1482] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:17:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:53.420930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e8e84-a1db-4db8-8afc-c6d6c73c23db None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:53.421361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e8e84-a1db-4db8-8afc-c6d6c73c23db None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:53.421492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9e8e84-a1db-4db8-8afc-c6d6c73c23db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:53.490545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cf9e8e84-a1db-4db8-8afc-c6d6c73c23db None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:53.491677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cf9e8e84-a1db-4db8-8afc-c6d6c73c23db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:53.540346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:53.592157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 736/1483] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:53.629886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d05a2-5a48-4c49-80a3-41ed16e812ec None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:53.631435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9e2d05a2-5a48-4c49-80a3-41ed16e812ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:53.655261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:53.736607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e2d05a2-5a48-4c49-80a3-41ed16e812ec None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:53.810698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fcdecc4-6625-478b-8707-ab99d4191f44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:53.811260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdecc4-6625-478b-8707-ab99d4191f44 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:53.811544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdecc4-6625-478b-8707-ab99d4191f44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:53.811820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fcdecc4-6625-478b-8707-ab99d4191f44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:53.813462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5fcdecc4-6625-478b-8707-ab99d4191f44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:53.889699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:53.924608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdecc4-6625-478b-8707-ab99d4191f44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:53.926078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fcdecc4-6625-478b-8707-ab99d4191f44 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:53.944071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 748/1484] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:56.043943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336981c-3416-4252-8732-a3a49d9b372a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:56.044503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336981c-3416-4252-8732-a3a49d9b372a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:56.044503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7336981c-3416-4252-8732-a3a49d9b372a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:56.045834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 737/1485] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:17:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:56.051168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e6577-9393-453f-aa46-ad9f975da0c5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:56.051633 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e6577-9393-453f-aa46-ad9f975da0c5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:56.051741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151e6577-9393-453f-aa46-ad9f975da0c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:56.116349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-151e6577-9393-453f-aa46-ad9f975da0c5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:56.117691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-151e6577-9393-453f-aa46-ad9f975da0c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:56.212785 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 749/1486] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:56.251433 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336981c-3416-4252-8732-a3a49d9b372a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:56.253001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7336981c-3416-4252-8732-a3a49d9b372a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:56.357585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7336981c-3416-4252-8732-a3a49d9b372a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:56.427899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:56.429831 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8105cef8-c312-4da8-93f7-6383f1a94ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:56.430495 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105cef8-c312-4da8-93f7-6383f1a94ffa service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:56.431034 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105cef8-c312-4da8-93f7-6383f1a94ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:56.431502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8105cef8-c312-4da8-93f7-6383f1a94ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:56.433711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8105cef8-c312-4da8-93f7-6383f1a94ffa service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:56.544889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105cef8-c312-4da8-93f7-6383f1a94ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:56.546362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8105cef8-c312-4da8-93f7-6383f1a94ffa service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:56.564077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 738/1487] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:17:58.771841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4615b54-e3fd-4785-8a9c-b872c35586a3 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:58.772368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4615b54-e3fd-4785-8a9c-b872c35586a3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:58.772368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4615b54-e3fd-4785-8a9c-b872c35586a3 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:58.774207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 750/1488] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:17:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:17:58.779417 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd44f8-1507-4b8a-b664-04c33bc7f307 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:58.779852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd44f8-1507-4b8a-b664-04c33bc7f307 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:58.780010 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fd44f8-1507-4b8a-b664-04c33bc7f307 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:58.845541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e4fd44f8-1507-4b8a-b664-04c33bc7f307 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:17:58.846790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e4fd44f8-1507-4b8a-b664-04c33bc7f307 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:58.899752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:58.944474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 739/1489] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:17:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:17:58.987185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4615b54-e3fd-4785-8a9c-b872c35586a3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:17:58.988761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b4615b54-e3fd-4785-8a9c-b872c35586a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:59.029279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:59.098250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4615b54-e3fd-4785-8a9c-b872c35586a3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:17:59.171789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee8e7318-025d-4f5a-9128-7a45d2346a35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:17:59.172440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e7318-025d-4f5a-9128-7a45d2346a35 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:17:59.172864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e7318-025d-4f5a-9128-7a45d2346a35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:17:59.173144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee8e7318-025d-4f5a-9128-7a45d2346a35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:17:59.174818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ee8e7318-025d-4f5a-9128-7a45d2346a35 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:17:59.255197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:17:59.284716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e7318-025d-4f5a-9128-7a45d2346a35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:17:59.286108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee8e7318-025d-4f5a-9128-7a45d2346a35 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:17:59.306089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 751/1490] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:17:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:18:01.583902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812ec01-0057-41a9-a224-f27e99ca46bb None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:01.584436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812ec01-0057-41a9-a224-f27e99ca46bb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:01.584436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812ec01-0057-41a9-a224-f27e99ca46bb None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:01.585871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 740/1491] 10.222.0.69 () {54 vars in 926 bytes} [Wed Jul 20 19:18:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:18:01.591972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347828f1-da83-42f0-bd6a-2f58d4e12696 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:01.592407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347828f1-da83-42f0-bd6a-2f58d4e12696 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:01.592555 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347828f1-da83-42f0-bd6a-2f58d4e12696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:01.659460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-347828f1-da83-42f0-bd6a-2f58d4e12696 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:18:01.660674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-347828f1-da83-42f0-bd6a-2f58d4e12696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:01.759275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 752/1492] 10.222.0.69 () {58 vars in 1023 bytes} [Wed Jul 20 19:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:18:01.798773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812ec01-0057-41a9-a224-f27e99ca46bb None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:18:01.800602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2812ec01-0057-41a9-a224-f27e99ca46bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:01.906386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812ec01-0057-41a9-a224-f27e99ca46bb None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:18:01.980538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ece5bf47-9dd7-47d2-99aa-5cb5bac05a07 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:18:01.981183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5bf47-9dd7-47d2-99aa-5cb5bac05a07 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:18:01.981464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5bf47-9dd7-47d2-99aa-5cb5bac05a07 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:18:01.981778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ece5bf47-9dd7-47d2-99aa-5cb5bac05a07 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:18:01.983441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ece5bf47-9dd7-47d2-99aa-5cb5bac05a07 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:18:01.992462 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:02.099479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5bf47-9dd7-47d2-99aa-5cb5bac05a07 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:18:02.100959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ece5bf47-9dd7-47d2-99aa-5cb5bac05a07 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:18:02.119912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 741/1493] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:18:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:18:11.757312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=82 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:18:12.117665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=50 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:36.683064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5c3f7-8913-456d-b9a3-a71ea93e5d61 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:36.683653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5c3f7-8913-456d-b9a3-a71ea93e5d61 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:36.683653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a5c3f7-8913-456d-b9a3-a71ea93e5d61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:36.764210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-08a5c3f7-8913-456d-b9a3-a71ea93e5d61 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:36.765409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-08a5c3f7-8913-456d-b9a3-a71ea93e5d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:36.862783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 753/1494] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:36.871122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee3d9e-e980-4c33-a73a-0d626c1f6934 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:36.871636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee3d9e-e980-4c33-a73a-0d626c1f6934 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:36.871746 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ee3d9e-e980-4c33-a73a-0d626c1f6934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:36.945351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b3ee3d9e-e980-4c33-a73a-0d626c1f6934 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:36.946525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b3ee3d9e-e980-4c33-a73a-0d626c1f6934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:37.042030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 742/1495] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:19:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:37.046718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5c3f7-8913-456d-b9a3-a71ea93e5d61 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:37.047963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-08a5c3f7-8913-456d-b9a3-a71ea93e5d61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:37.059464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a5c3f7-8913-456d-b9a3-a71ea93e5d61 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:37.120272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:37.154801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:37.155462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:37.155752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:37.156031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:37.156749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:19:37.157058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:37.158104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:37.158104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 20 19:19:37.158104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: Jul 20 19:19:37.158997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:37.170797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 754/1496] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:19:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:19:37.179084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee3d9e-e980-4c33-a73a-0d626c1f6934 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:37.188274 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ee3d9e-e980-4c33-a73a-0d626c1f6934 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:37.249758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:37.287935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97c71d-60c4-4425-8596-ec8b4847c3cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:37.288606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97c71d-60c4-4425-8596-ec8b4847c3cb admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:37.288897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97c71d-60c4-4425-8596-ec8b4847c3cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:37.289172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c97c71d-60c4-4425-8596-ec8b4847c3cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:37.289918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97c71d-60c4-4425-8596-ec8b4847c3cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:37.291737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-6c97c71d-60c4-4425-8596-ec8b4847c3cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:37.291737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 20 19:19:37.291737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: Jul 20 19:19:37.292962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c97c71d-60c4-4425-8596-ec8b4847c3cb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:37.349932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 743/1497] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:19:37 2022] POST /identity/v3/projects => generated 379 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:19:37.354358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:37.355986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:37.389898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf2090f-5042-4994-85d7-4ae9507eb8d9 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:37.484694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152c56-5182-4ed3-8d60-a10e0962e2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:37.485356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152c56-5182-4ed3-8d60-a10e0962e2dd admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:37.485668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152c56-5182-4ed3-8d60-a10e0962e2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:37.485947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7152c56-5182-4ed3-8d60-a10e0962e2dd admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:37.486642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7152c56-5182-4ed3-8d60-a10e0962e2dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:37.487651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-b7152c56-5182-4ed3-8d60-a10e0962e2dd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:37.487651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 20 19:19:37.487651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: Jul 20 19:19:37.488859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7152c56-5182-4ed3-8d60-a10e0962e2dd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:37.530306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 755/1498] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:19:37 2022] POST /identity/v3/users => generated 290 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:19:37.534517 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97c71d-60c4-4425-8596-ec8b4847c3cb admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:37.536205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6c97c71d-60c4-4425-8596-ec8b4847c3cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:37.600533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c97c71d-60c4-4425-8596-ec8b4847c3cb admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:37.687451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:37.698564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b2959-7ddb-4b08-af93-aa0cf53e7ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:37.699243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b2959-7ddb-4b08-af93-aa0cf53e7ec5 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:37.699557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b2959-7ddb-4b08-af93-aa0cf53e7ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:37.699838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0b2959-7ddb-4b08-af93-aa0cf53e7ec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:37.700572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b2959-7ddb-4b08-af93-aa0cf53e7ec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:37.701618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-2e0b2959-7ddb-4b08-af93-aa0cf53e7ec5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:37.701618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 20 19:19:37.701618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: Jul 20 19:19:37.702542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e0b2959-7ddb-4b08-af93-aa0cf53e7ec5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:37.715517 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 744/1499] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:19:37.719895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152c56-5182-4ed3-8d60-a10e0962e2dd admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:37.721460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b7152c56-5182-4ed3-8d60-a10e0962e2dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:37.733732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7152c56-5182-4ed3-8d60-a10e0962e2dd admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:37.759222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:37.830252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1616c6de-91fe-445d-b6b0-96aedf0428c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:37.830839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616c6de-91fe-445d-b6b0-96aedf0428c5 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:37.831124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616c6de-91fe-445d-b6b0-96aedf0428c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:37.831400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1616c6de-91fe-445d-b6b0-96aedf0428c5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:37.832126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616c6de-91fe-445d-b6b0-96aedf0428c5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:37.833137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-1616c6de-91fe-445d-b6b0-96aedf0428c5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:37.833137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 20 19:19:37.833137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: Jul 20 19:19:37.834578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1616c6de-91fe-445d-b6b0-96aedf0428c5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:37.847147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 756/1500] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:19:37 2022] GET /identity/v3/roles => generated 1420 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:19:37.852050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b2959-7ddb-4b08-af93-aa0cf53e7ec5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:37.853616 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2e0b2959-7ddb-4b08-af93-aa0cf53e7ec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:37.887107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e0b2959-7ddb-4b08-af93-aa0cf53e7ec5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:37.982047 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b2a5a-1e69-45ed-bdc1-5d727f682622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:37.982941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b2a5a-1e69-45ed-bdc1-5d727f682622 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:37.983223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b2a5a-1e69-45ed-bdc1-5d727f682622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:37.983503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b2a5a-1e69-45ed-bdc1-5d727f682622 admin admin] PATH_INFO: `/v3/projects/adf5d338b83c49439edf721a549e5fca/users/aba7da5f53954fb483ac49030807e4e6/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:38.005332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b2a5a-1e69-45ed-bdc1-5d727f682622 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adf5d338b83c49439edf721a549e5fca, user_id=aba7da5f53954fb483ac49030807e4e6, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:38.007152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-5e3b2a5a-1e69-45ed-bdc1-5d727f682622 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:38.007152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 20 19:19:38.007152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: Jul 20 19:19:38.008936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b2a5a-1e69-45ed-bdc1-5d727f682622 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:38.035040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 745/1501] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:19:37 2022] PUT /identity/v3/projects/adf5d338b83c49439edf721a549e5fca/users/aba7da5f53954fb483ac49030807e4e6/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:19:38.047104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be376d56-6c23-473a-9fbf-8ababe5dd8f5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:38.047397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be376d56-6c23-473a-9fbf-8ababe5dd8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:38.047677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be376d56-6c23-473a-9fbf-8ababe5dd8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:38.103771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-be376d56-6c23-473a-9fbf-8ababe5dd8f5 None None] MFA Rules not processed for user `aba7da5f53954fb483ac49030807e4e6`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:38.104894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-be376d56-6c23-473a-9fbf-8ababe5dd8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:38.180987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 757/1502] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:19:38 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:38.194722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20f28c-a669-4acb-bbf9-0917e4456f5b None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:38.195008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20f28c-a669-4acb-bbf9-0917e4456f5b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:38.195311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f20f28c-a669-4acb-bbf9-0917e4456f5b None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:38.196711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 746/1503] 10.222.0.69 () {54 vars in 964 bytes} [Wed Jul 20 19:19:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:38.201614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffefbcf-8fa1-49f3-b86e-18fd9810a1c4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:38.201914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffefbcf-8fa1-49f3-b86e-18fd9810a1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:38.202196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ffefbcf-8fa1-49f3-b86e-18fd9810a1c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:38.232449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:38.278153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2ffefbcf-8fa1-49f3-b86e-18fd9810a1c4 None None] MFA Rules not processed for user `c94d9030f61a4e1f8b3d75ef8925bdd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:38.279353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2ffefbcf-8fa1-49f3-b86e-18fd9810a1c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:38.356295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 758/1504] 10.222.0.69 () {58 vars in 1061 bytes} [Wed Jul 20 19:19:38 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:38.363922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20f28c-a669-4acb-bbf9-0917e4456f5b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:38.366017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3f20f28c-a669-4acb-bbf9-0917e4456f5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:38.415749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:38.472185 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f20f28c-a669-4acb-bbf9-0917e4456f5b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:38.549228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c245a-4b2f-4a7d-b1b2-5381b5e41926 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:38.549916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c245a-4b2f-4a7d-b1b2-5381b5e41926 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:38.550202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c245a-4b2f-4a7d-b1b2-5381b5e41926 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:38.550482 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-096c245a-4b2f-4a7d-b1b2-5381b5e41926 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:38.552143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-096c245a-4b2f-4a7d-b1b2-5381b5e41926 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:38.652899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c245a-4b2f-4a7d-b1b2-5381b5e41926 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:38.654376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-096c245a-4b2f-4a7d-b1b2-5381b5e41926 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:38.657904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:38.673397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 747/1505] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:38 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:38.716784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf62b9-f42d-4cbd-8f92-8aa41c9fde7c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:38.717074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf62b9-f42d-4cbd-8f92-8aa41c9fde7c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:38.717353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf62b9-f42d-4cbd-8f92-8aa41c9fde7c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:38.718685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 759/1506] 10.222.0.69 () {52 vars in 829 bytes} [Wed Jul 20 19:19:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:19:38.722472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c245a-4b2f-4a7d-b1b2-5381b5e41926 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:38.723687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-096c245a-4b2f-4a7d-b1b2-5381b5e41926 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:38.823817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-096c245a-4b2f-4a7d-b1b2-5381b5e41926 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:38.917723 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:38.918514 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:38.918796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:38.919100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] PATH_INFO: `/v3/users/aba7da5f53954fb483ac49030807e4e6` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:38.932159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=aba7da5f53954fb483ac49030807e4e6)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:38.933190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:38.933190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 20 19:19:38.933190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: Jul 20 19:19:38.934462 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:38.944871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:39.001498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aba7da5f53954fb483ac49030807e4e6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:39.020245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aba7da5f53954fb483ac49030807e4e6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:39.032798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aba7da5f53954fb483ac49030807e4e6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:39.033371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aba7da5f53954fb483ac49030807e4e6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:39.033827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aba7da5f53954fb483ac49030807e4e6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:39.048252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 748/1507] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:19:38 2022] DELETE /identity/v3/users/aba7da5f53954fb483ac49030807e4e6 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:19:39.053186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1dab4-0e5f-40ab-ac5e-cd43861f0e92 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:39.053685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1dab4-0e5f-40ab-ac5e-cd43861f0e92 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:39.054013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af1dab4-0e5f-40ab-ac5e-cd43861f0e92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:39.108098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8af1dab4-0e5f-40ab-ac5e-cd43861f0e92 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:19:39.109307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8af1dab4-0e5f-40ab-ac5e-cd43861f0e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:39.183432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:39.198554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 760/1508] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:19:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:19:39.207984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:39.209248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:39.430561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f03f4-8303-4d0c-9f1f-5dc490c436d7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:39.485194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:39.506763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052f4b14-54f0-42f2-8d8f-332220e41012 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:39.507426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f4b14-54f0-42f2-8d8f-332220e41012 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:39.507707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f4b14-54f0-42f2-8d8f-332220e41012 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:39.507981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052f4b14-54f0-42f2-8d8f-332220e41012 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:39.509692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-052f4b14-54f0-42f2-8d8f-332220e41012 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:39.620068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f4b14-54f0-42f2-8d8f-332220e41012 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:39.621501 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052f4b14-54f0-42f2-8d8f-332220e41012 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:39.639449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 749/1509] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:19:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 432 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:19:39.969197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1dab4-0e5f-40ab-ac5e-cd43861f0e92 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:19:39.970443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8af1dab4-0e5f-40ab-ac5e-cd43861f0e92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:19:39.983577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8af1dab4-0e5f-40ab-ac5e-cd43861f0e92 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:19:40.083109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:19:40.084386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:19:40.084840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:19:40.085240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] PATH_INFO: `/v3/projects/adf5d338b83c49439edf721a549e5fca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:19:40.095591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adf5d338b83c49439edf721a549e5fca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:19:40.096610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:19:40.096610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 20 19:19:40.096610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: Jul 20 19:19:40.097842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:19:40.156376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adf5d338b83c49439edf721a549e5fca', 'request_id': 'req-1020ed1f-c758-44a1-a770-56783999a240'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:40.170387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adf5d338b83c49439edf721a549e5fca', 'request_id': 'req-1020ed1f-c758-44a1-a770-56783999a240'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:40.176362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adf5d338b83c49439edf721a549e5fca', 'request_id': 'req-1020ed1f-c758-44a1-a770-56783999a240'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:40.203691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] The token cache is being invalidate because project adf5d338b83c49439edf721a549e5fca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:19:40.203985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:40.204498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1020ed1f-c758-44a1-a770-56783999a240 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:19:40.205889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 761/1510] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:19:39 2022] DELETE /identity/v3/projects/adf5d338b83c49439edf721a549e5fca => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:19:49.637183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=62 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:19:50.201647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:25.922265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3aec9-ca48-4f0d-b8f1-5636ba1a71e5 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:25.922895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3aec9-ca48-4f0d-b8f1-5636ba1a71e5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:25.922895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ff3aec9-ca48-4f0d-b8f1-5636ba1a71e5 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:25.924184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 750/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:21:25.928952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a217f-7337-430a-9783-b0ce6b0f461d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:25.929480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a217f-7337-430a-9783-b0ce6b0f461d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:25.929611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d2a217f-7337-430a-9783-b0ce6b0f461d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:26.009699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0d2a217f-7337-430a-9783-b0ce6b0f461d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:26.010919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0d2a217f-7337-430a-9783-b0ce6b0f461d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:26.116410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 762/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:26.271610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3aec9-ca48-4f0d-b8f1-5636ba1a71e5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:26.273825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8ff3aec9-ca48-4f0d-b8f1-5636ba1a71e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:26.392000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ff3aec9-ca48-4f0d-b8f1-5636ba1a71e5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:26.397361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d3ecef-3ad6-4022-8762-06e669edc5e6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:26.398065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3ecef-3ad6-4022-8762-06e669edc5e6 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:26.398343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3ecef-3ad6-4022-8762-06e669edc5e6 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:26.398617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d3ecef-3ad6-4022-8762-06e669edc5e6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:26.406689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3ecef-3ad6-4022-8762-06e669edc5e6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:26.408268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d3ecef-3ad6-4022-8762-06e669edc5e6 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:26.417724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 751/1513] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:27.271364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a9ca6-fe0d-4215-a2eb-ad18e85f13a8 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:27.272312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a9ca6-fe0d-4215-a2eb-ad18e85f13a8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:27.272604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c46a9ca6-fe0d-4215-a2eb-ad18e85f13a8 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:27.274117 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 763/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:27.280160 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8dfd-c833-46ff-8528-c99c22044415 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:27.280710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8dfd-c833-46ff-8528-c99c22044415 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:27.280710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8df8dfd-c833-46ff-8528-c99c22044415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:27.299239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b8df8dfd-c833-46ff-8528-c99c22044415 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:27.300474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b8df8dfd-c833-46ff-8528-c99c22044415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:27.309861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 752/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:27.467679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a9ca6-fe0d-4215-a2eb-ad18e85f13a8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:27.482751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c46a9ca6-fe0d-4215-a2eb-ad18e85f13a8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:27.488168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-010e9275-743b-4090-a7a4-d4b42eba541e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:27.488803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e9275-743b-4090-a7a4-d4b42eba541e service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:27.489083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e9275-743b-4090-a7a4-d4b42eba541e service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:27.489358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-010e9275-743b-4090-a7a4-d4b42eba541e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:27.497713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e9275-743b-4090-a7a4-d4b42eba541e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:27.499224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-010e9275-743b-4090-a7a4-d4b42eba541e service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:27.509739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 764/1516] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:31.238915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58180ce-2062-47fe-b534-a39e60d9e964 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:31.239533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58180ce-2062-47fe-b534-a39e60d9e964 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:31.239533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58180ce-2062-47fe-b534-a39e60d9e964 None None] PATH_INFO: `/v3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:31.240784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 753/1517] 10.222.0.93 () {54 vars in 925 bytes} [Wed Jul 20 19:21:31 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:21:31.247769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205af28f-a4a6-4ae6-aa61-8dbc7b66adfd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:31.248178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205af28f-a4a6-4ae6-aa61-8dbc7b66adfd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:31.248324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205af28f-a4a6-4ae6-aa61-8dbc7b66adfd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:31.308085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-205af28f-a4a6-4ae6-aa61-8dbc7b66adfd None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:31.309294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-205af28f-a4a6-4ae6-aa61-8dbc7b66adfd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:31.326424 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:31.381813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 765/1518] 10.222.0.93 () {58 vars in 1015 bytes} [Wed Jul 20 19:21:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:35.742213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c222a14-b558-469c-a491-36e3729ea46a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:35.742949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c222a14-b558-469c-a491-36e3729ea46a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:35.742949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c222a14-b558-469c-a491-36e3729ea46a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:35.744259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 754/1519] 10.222.0.69 () {54 vars in 961 bytes} [Wed Jul 20 19:21:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:35.749153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67e37f-e850-4324-8fc7-a7961a27de84 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:35.749619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67e37f-e850-4324-8fc7-a7961a27de84 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:35.749733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c67e37f-e850-4324-8fc7-a7961a27de84 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:35.768356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4c67e37f-e850-4324-8fc7-a7961a27de84 None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:35.769563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4c67e37f-e850-4324-8fc7-a7961a27de84 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:35.778738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 766/1520] 10.222.0.69 () {58 vars in 1058 bytes} [Wed Jul 20 19:21:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:35.786094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c222a14-b558-469c-a491-36e3729ea46a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:35.799504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c222a14-b558-469c-a491-36e3729ea46a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:35.804374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d9abd6-3349-48bc-a4cd-4b5ea5623650 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:35.805102 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9abd6-3349-48bc-a4cd-4b5ea5623650 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:35.805424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9abd6-3349-48bc-a4cd-4b5ea5623650 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:35.805739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d9abd6-3349-48bc-a4cd-4b5ea5623650 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:35.814228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9abd6-3349-48bc-a4cd-4b5ea5623650 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:35.815808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d9abd6-3349-48bc-a4cd-4b5ea5623650 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:35.825875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 755/1521] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:38.721266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faca28e-ed86-4d64-8bf1-df0bad754a20 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:38.721854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faca28e-ed86-4d64-8bf1-df0bad754a20 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:38.721854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8faca28e-ed86-4d64-8bf1-df0bad754a20 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:38.723241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 767/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:38.728715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f2a5d-bca9-482c-aa30-bd9779c78ec7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:38.729298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f2a5d-bca9-482c-aa30-bd9779c78ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:38.729298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2f2a5d-bca9-482c-aa30-bd9779c78ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:38.748085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4d2f2a5d-bca9-482c-aa30-bd9779c78ec7 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:38.749277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4d2f2a5d-bca9-482c-aa30-bd9779c78ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:38.757109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 756/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:38.911769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faca28e-ed86-4d64-8bf1-df0bad754a20 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:38.926662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8faca28e-ed86-4d64-8bf1-df0bad754a20 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:38.932471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ff671e-a95b-4e70-aa13-63a15ce65413 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:38.933112 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff671e-a95b-4e70-aa13-63a15ce65413 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:38.933392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff671e-a95b-4e70-aa13-63a15ce65413 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:38.933715 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ff671e-a95b-4e70-aa13-63a15ce65413 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:38.942023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff671e-a95b-4e70-aa13-63a15ce65413 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:38.943597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57ff671e-a95b-4e70-aa13-63a15ce65413 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:38.952990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 768/1524] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:39.795607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e420d1-e2bc-4b54-aced-925762fc2ca4 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:39.796189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e420d1-e2bc-4b54-aced-925762fc2ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:39.796189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e420d1-e2bc-4b54-aced-925762fc2ca4 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:39.797643 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 757/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 20 19:21:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:39.802569 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4c648-8f40-4411-9dae-6a958be5fbb2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:39.803014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4c648-8f40-4411-9dae-6a958be5fbb2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:39.803115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef4c648-8f40-4411-9dae-6a958be5fbb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:39.822172 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6ef4c648-8f40-4411-9dae-6a958be5fbb2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:39.823372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6ef4c648-8f40-4411-9dae-6a958be5fbb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:39.831310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 769/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 20 19:21:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:39.985238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e420d1-e2bc-4b54-aced-925762fc2ca4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:40.000165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e420d1-e2bc-4b54-aced-925762fc2ca4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:40.005719 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1db6d4f-c136-440d-b52d-57738dd093a0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:40.006354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db6d4f-c136-440d-b52d-57738dd093a0 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:40.006639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db6d4f-c136-440d-b52d-57738dd093a0 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:40.006917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1db6d4f-c136-440d-b52d-57738dd093a0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:40.015208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db6d4f-c136-440d-b52d-57738dd093a0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:40.016796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1db6d4f-c136-440d-b52d-57738dd093a0 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:40.026230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 758/1527] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:44.030495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba094c7-92ef-4d1c-84d1-2969dd1e2470 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:44.031085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba094c7-92ef-4d1c-84d1-2969dd1e2470 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:44.031085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba094c7-92ef-4d1c-84d1-2969dd1e2470 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:44.032426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 770/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:44.037716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca6d25-5298-4afb-b1c0-b3d0e471e764 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:44.038139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca6d25-5298-4afb-b1c0-b3d0e471e764 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:44.038278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fca6d25-5298-4afb-b1c0-b3d0e471e764 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:44.057172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7fca6d25-5298-4afb-b1c0-b3d0e471e764 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:44.058407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7fca6d25-5298-4afb-b1c0-b3d0e471e764 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:44.066380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 759/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:44.141917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba094c7-92ef-4d1c-84d1-2969dd1e2470 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:44.143608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6ba094c7-92ef-4d1c-84d1-2969dd1e2470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:44.223954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba094c7-92ef-4d1c-84d1-2969dd1e2470 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:44.227542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9e5715-9679-475b-b526-bf3b9d63d16b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:44.228185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e5715-9679-475b-b526-bf3b9d63d16b service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:44.228470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e5715-9679-475b-b526-bf3b9d63d16b service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:44.228749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9e5715-9679-475b-b526-bf3b9d63d16b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:44.237719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e5715-9679-475b-b526-bf3b9d63d16b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:44.239179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9e5715-9679-475b-b526-bf3b9d63d16b service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:44.249167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 771/1530] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:46.283287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33170963-bef8-4542-ad75-4605db5f7d6f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:46.283899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33170963-bef8-4542-ad75-4605db5f7d6f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:46.283899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33170963-bef8-4542-ad75-4605db5f7d6f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:46.285235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 760/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:21:46.291421 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e15cda-c82e-49e2-b737-8a7d7ea31a0d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:46.291886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e15cda-c82e-49e2-b737-8a7d7ea31a0d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:46.291998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e15cda-c82e-49e2-b737-8a7d7ea31a0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:46.310973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-32e15cda-c82e-49e2-b737-8a7d7ea31a0d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:46.312206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-32e15cda-c82e-49e2-b737-8a7d7ea31a0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:46.320178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 772/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:46.358657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33170963-bef8-4542-ad75-4605db5f7d6f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:46.369358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33170963-bef8-4542-ad75-4605db5f7d6f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:46.372824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bf5e8-7c94-435c-bb32-c9d74b457e6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:46.373450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bf5e8-7c94-435c-bb32-c9d74b457e6a service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:46.373766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bf5e8-7c94-435c-bb32-c9d74b457e6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:46.374019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bf5e8-7c94-435c-bb32-c9d74b457e6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:46.382679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850bf5e8-7c94-435c-bb32-c9d74b457e6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:46.384452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850bf5e8-7c94-435c-bb32-c9d74b457e6a service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:46.393885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 761/1533] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:50.161261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e04c1-ef68-4ccb-b477-0ab768d653b6 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:50.161878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e04c1-ef68-4ccb-b477-0ab768d653b6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:50.161878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013e04c1-ef68-4ccb-b477-0ab768d653b6 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:50.163281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 773/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:50.169450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c31d71-0ee1-4662-a5d6-1b1339cf777e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:50.169953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c31d71-0ee1-4662-a5d6-1b1339cf777e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:50.170064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c31d71-0ee1-4662-a5d6-1b1339cf777e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:50.176227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=81 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:50.189031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a0c31d71-0ee1-4662-a5d6-1b1339cf777e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:50.190276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a0c31d71-0ee1-4662-a5d6-1b1339cf777e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:50.198314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 762/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:50.505099 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e04c1-ef68-4ccb-b477-0ab768d653b6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:50.506796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-013e04c1-ef68-4ccb-b477-0ab768d653b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:50.573013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:21:50.586554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013e04c1-ef68-4ccb-b477-0ab768d653b6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:50.591000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7c980c-bd33-4fc4-8394-0de094e0ef3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:50.591645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c980c-bd33-4fc4-8394-0de094e0ef3a service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:50.591930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c980c-bd33-4fc4-8394-0de094e0ef3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:50.592207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7c980c-bd33-4fc4-8394-0de094e0ef3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:50.601583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c980c-bd33-4fc4-8394-0de094e0ef3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:50.603017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7c980c-bd33-4fc4-8394-0de094e0ef3a service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:50.612479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 774/1536] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:21:52.709919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15080fa-7169-4881-8eb5-45b292edb1ee None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:52.710493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15080fa-7169-4881-8eb5-45b292edb1ee None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:52.710493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15080fa-7169-4881-8eb5-45b292edb1ee None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:52.712435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 763/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 20 19:21:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:21:52.717960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0d27-46b2-4fe0-bd1d-740a4decce0f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:52.718394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0d27-46b2-4fe0-bd1d-740a4decce0f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:52.718523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db0d27-46b2-4fe0-bd1d-740a4decce0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:52.736477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-73db0d27-46b2-4fe0-bd1d-740a4decce0f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:21:52.737872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-73db0d27-46b2-4fe0-bd1d-740a4decce0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:52.746185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 775/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 20 19:21:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:21:53.055462 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15080fa-7169-4881-8eb5-45b292edb1ee None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:21:53.057953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f15080fa-7169-4881-8eb5-45b292edb1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:21:53.071060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15080fa-7169-4881-8eb5-45b292edb1ee None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:21:53.075161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-566b6f34-0671-4e1b-9df2-55e95f0208d6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:21:53.075804 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b6f34-0671-4e1b-9df2-55e95f0208d6 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:21:53.076089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b6f34-0671-4e1b-9df2-55e95f0208d6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:21:53.076376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-566b6f34-0671-4e1b-9df2-55e95f0208d6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:21:53.085254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b6f34-0671-4e1b-9df2-55e95f0208d6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:21:53.086724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-566b6f34-0671-4e1b-9df2-55e95f0208d6 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:21:53.096701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 764/1539] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:21:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:22:02.731475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=11 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:22:03.091989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=8 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:29.484722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f720d6a-742d-4af5-85ff-f598b843eecd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.485358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f720d6a-742d-4af5-85ff-f598b843eecd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.485358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f720d6a-742d-4af5-85ff-f598b843eecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.510757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2f720d6a-742d-4af5-85ff-f598b843eecd None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:29.512196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2f720d6a-742d-4af5-85ff-f598b843eecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:29.523243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 776/1540] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:29.533850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e0e15-dff1-494d-ada8-755b32a6cea5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.534455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e0e15-dff1-494d-ada8-755b32a6cea5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.534455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656e0e15-dff1-494d-ada8-755b32a6cea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.559052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-656e0e15-dff1-494d-ada8-755b32a6cea5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:29.560369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-656e0e15-dff1-494d-ada8-755b32a6cea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:29.578327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 765/1541] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:29.582660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f720d6a-742d-4af5-85ff-f598b843eecd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:29.599734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f720d6a-742d-4af5-85ff-f598b843eecd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:29.608215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cc469b3-1797-4ef5-999d-3368af692382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:29.608923 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc469b3-1797-4ef5-999d-3368af692382 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.609213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc469b3-1797-4ef5-999d-3368af692382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.609497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc469b3-1797-4ef5-999d-3368af692382 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.610283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc469b3-1797-4ef5-999d-3368af692382 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:29.610600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc469b3-1797-4ef5-999d-3368af692382 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:29.612286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cc469b3-1797-4ef5-999d-3368af692382 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:29.630414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 777/1542] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:29.639581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e0e15-dff1-494d-ada8-755b32a6cea5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:29.648645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5aabb-ab46-4c44-923e-0cc6c375be11 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.649021 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5aabb-ab46-4c44-923e-0cc6c375be11 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.649387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a5aabb-ab46-4c44-923e-0cc6c375be11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.653600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656e0e15-dff1-494d-ada8-755b32a6cea5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:29.665076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169239b8-3d34-4e35-94db-6b940ad46f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:29.665810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169239b8-3d34-4e35-94db-6b940ad46f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.666163 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169239b8-3d34-4e35-94db-6b940ad46f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.666451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-169239b8-3d34-4e35-94db-6b940ad46f49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.669380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169239b8-3d34-4e35-94db-6b940ad46f49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:29.672644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-169239b8-3d34-4e35-94db-6b940ad46f49 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:29.702790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c9a5aabb-ab46-4c44-923e-0cc6c375be11 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:29.704070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c9a5aabb-ab46-4c44-923e-0cc6c375be11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:29.725360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 778/1543] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:29.737381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5273e0-5955-4258-a295-4f45e70938d0 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.737773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5273e0-5955-4258-a295-4f45e70938d0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.738032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5273e0-5955-4258-a295-4f45e70938d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.759973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 766/1544] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:29 2022] POST /identity/v3/projects => generated 377 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:29.766787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169239b8-3d34-4e35-94db-6b940ad46f49 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:29.780348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bf5273e0-5955-4258-a295-4f45e70938d0 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:29.788331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bf5273e0-5955-4258-a295-4f45e70938d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:29.890465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-169239b8-3d34-4e35-94db-6b940ad46f49 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:29.896226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb88857d-baed-47d9-b082-0b6729bc3eae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:29.897286 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88857d-baed-47d9-b082-0b6729bc3eae admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.897910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88857d-baed-47d9-b082-0b6729bc3eae admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.898494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb88857d-baed-47d9-b082-0b6729bc3eae admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.899436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88857d-baed-47d9-b082-0b6729bc3eae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:29.900954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-eb88857d-baed-47d9-b082-0b6729bc3eae admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:29.900954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 20 19:23:29.900954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: Jul 20 19:23:29.902458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 779/1545] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:29.902960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb88857d-baed-47d9-b082-0b6729bc3eae admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:29.907306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8fad-8885-4343-bdf7-23fc77cd0f6a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.907306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8fad-8885-4343-bdf7-23fc77cd0f6a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.907306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0a8fad-8885-4343-bdf7-23fc77cd0f6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.929222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2b0a8fad-8885-4343-bdf7-23fc77cd0f6a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:29.932499 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2b0a8fad-8885-4343-bdf7-23fc77cd0f6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:29.940095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 767/1546] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:29 2022] POST /identity/v3/users => generated 289 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:29.941550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca60f0a-e3ab-45ff-a424-c5ff8ea9f5fc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.941857 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca60f0a-e3ab-45ff-a424-c5ff8ea9f5fc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.942138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca60f0a-e3ab-45ff-a424-c5ff8ea9f5fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.942420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 780/1547] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:29.943927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37792ae9-7268-45ef-bfa5-3d69b0272d5f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.944377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37792ae9-7268-45ef-bfa5-3d69b0272d5f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.944820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37792ae9-7268-45ef-bfa5-3d69b0272d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.963610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-eca60f0a-e3ab-45ff-a424-c5ff8ea9f5fc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:29.965043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eca60f0a-e3ab-45ff-a424-c5ff8ea9f5fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:29.965780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-37792ae9-7268-45ef-bfa5-3d69b0272d5f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:29.966964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-37792ae9-7268-45ef-bfa5-3d69b0272d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:29.975384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 781/1548] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:29.975918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 768/1549] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:29.976623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac2476-0021-4448-8e3b-89015c23b052 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.976749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca60f0a-e3ab-45ff-a424-c5ff8ea9f5fc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:29.976930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac2476-0021-4448-8e3b-89015c23b052 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.977191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72ac2476-0021-4448-8e3b-89015c23b052 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.989650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca60f0a-e3ab-45ff-a424-c5ff8ea9f5fc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:29.994830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb7077-9065-4436-b331-c2f6a626207d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:29.995508 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb7077-9065-4436-b331-c2f6a626207d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:29.995952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb7077-9065-4436-b331-c2f6a626207d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:29.996749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eb7077-9065-4436-b331-c2f6a626207d admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:29.997030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb7077-9065-4436-b331-c2f6a626207d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:29.997345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb7077-9065-4436-b331-c2f6a626207d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:29.997748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-72ac2476-0021-4448-8e3b-89015c23b052 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:30.001911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-83eb7077-9065-4436-b331-c2f6a626207d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:30.001911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 20 19:23:30.001911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: Jul 20 19:23:30.002213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-72ac2476-0021-4448-8e3b-89015c23b052 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:30.002296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eb7077-9065-4436-b331-c2f6a626207d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.013143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 782/1550] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:30.014933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc2523-f7e8-475b-ba57-595e5078e44f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.015382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc2523-f7e8-475b-ba57-595e5078e44f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.015847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58fc2523-f7e8-475b-ba57-595e5078e44f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.021539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 769/1551] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:30.021539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb7077-9065-4436-b331-c2f6a626207d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.033191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eb7077-9065-4436-b331-c2f6a626207d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.038035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d4db03-6fbb-4c70-8c02-4670add53395 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.038699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4db03-6fbb-4c70-8c02-4670add53395 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.038980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4db03-6fbb-4c70-8c02-4670add53395 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.039254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d4db03-6fbb-4c70-8c02-4670add53395 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.039361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-58fc2523-f7e8-475b-ba57-595e5078e44f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:30.040004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4db03-6fbb-4c70-8c02-4670add53395 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.040739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-58fc2523-f7e8-475b-ba57-595e5078e44f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:30.041736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d4db03-6fbb-4c70-8c02-4670add53395 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.049294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 783/1552] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:30.050601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43cef-b372-42b6-a89f-047b8cc993da None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.050891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43cef-b372-42b6-a89f-047b8cc993da None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.051189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf43cef-b372-42b6-a89f-047b8cc993da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.052938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 770/1553] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:30.054225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee15ad6-a9c0-4201-b5a6-2bc354b41566 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.054549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee15ad6-a9c0-4201-b5a6-2bc354b41566 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.054828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee15ad6-a9c0-4201-b5a6-2bc354b41566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.073345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0cf43cef-b372-42b6-a89f-047b8cc993da None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:30.074594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0cf43cef-b372-42b6-a89f-047b8cc993da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:30.080926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-aee15ad6-a9c0-4201-b5a6-2bc354b41566 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:30.082133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aee15ad6-a9c0-4201-b5a6-2bc354b41566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:30.082620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 784/1554] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:30.086573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988367a-bdd4-4efa-9744-71180b6a66e1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.086573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988367a-bdd4-4efa-9744-71180b6a66e1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.086573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f988367a-bdd4-4efa-9744-71180b6a66e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.091039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 771/1555] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:30.092308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19d095-aba9-42a1-9398-6a1501d622e2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.092596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19d095-aba9-42a1-9398-6a1501d622e2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.093136 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b19d095-aba9-42a1-9398-6a1501d622e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.104369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f988367a-bdd4-4efa-9744-71180b6a66e1 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:30.105618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f988367a-bdd4-4efa-9744-71180b6a66e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:30.111964 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-9b19d095-aba9-42a1-9398-6a1501d622e2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:30.113126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9b19d095-aba9-42a1-9398-6a1501d622e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:30.113623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 785/1556] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:30.114920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000dd92-c347-4ab9-a64e-cb7384ba8c95 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.115217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000dd92-c347-4ab9-a64e-cb7384ba8c95 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.115499 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e000dd92-c347-4ab9-a64e-cb7384ba8c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.126724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 772/1557] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:30.127914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19d095-aba9-42a1-9398-6a1501d622e2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.134078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e000dd92-c347-4ab9-a64e-cb7384ba8c95 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:30.135300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e000dd92-c347-4ab9-a64e-cb7384ba8c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:30.138114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b19d095-aba9-42a1-9398-6a1501d622e2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.142865 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6fbed6-5783-4d88-8d5c-c6f84169231f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.143339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 786/1558] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:30.143453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fbed6-5783-4d88-8d5c-c6f84169231f admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.143723 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fbed6-5783-4d88-8d5c-c6f84169231f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.144061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6fbed6-5783-4d88-8d5c-c6f84169231f admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.144165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000dd92-c347-4ab9-a64e-cb7384ba8c95 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.144759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fbed6-5783-4d88-8d5c-c6f84169231f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.146961 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6fbed6-5783-4d88-8d5c-c6f84169231f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.156621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e000dd92-c347-4ab9-a64e-cb7384ba8c95 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.163823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d40fa0-c48a-4c58-92a0-733314f50694 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.165458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d40fa0-c48a-4c58-92a0-733314f50694 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.165970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d40fa0-c48a-4c58-92a0-733314f50694 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.166429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d40fa0-c48a-4c58-92a0-733314f50694 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.167819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d40fa0-c48a-4c58-92a0-733314f50694 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.170566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d40fa0-c48a-4c58-92a0-733314f50694 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.182954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 773/1559] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/projects => generated 373 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:30.188952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157b62-ea68-4440-a78d-73e195cb83fb None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.188952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157b62-ea68-4440-a78d-73e195cb83fb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.188952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b157b62-ea68-4440-a78d-73e195cb83fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.189458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 787/1560] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:30.189458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d40fa0-c48a-4c58-92a0-733314f50694 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.199969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d40fa0-c48a-4c58-92a0-733314f50694 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.206935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5b157b62-ea68-4440-a78d-73e195cb83fb None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:30.208198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5b157b62-ea68-4440-a78d-73e195cb83fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:30.255114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:30.261787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c52b48-9e4f-4049-92b7-acd96dccfaa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.262653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c52b48-9e4f-4049-92b7-acd96dccfaa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.262961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c52b48-9e4f-4049-92b7-acd96dccfaa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.263278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c52b48-9e4f-4049-92b7-acd96dccfaa0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.264997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c52b48-9e4f-4049-92b7-acd96dccfaa0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:30.265319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c52b48-9e4f-4049-92b7-acd96dccfaa0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.266980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c52b48-9e4f-4049-92b7-acd96dccfaa0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.277505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 774/1561] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:30.278240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b157b62-ea68-4440-a78d-73e195cb83fb None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.281841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 788/1562] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:30.282573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c52b48-9e4f-4049-92b7-acd96dccfaa0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.288461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b157b62-ea68-4440-a78d-73e195cb83fb None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.296152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e5e3d7f-f007-4d07-bf4c-2d72edc456c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.296824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e3d7f-f007-4d07-bf4c-2d72edc456c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.297113 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e3d7f-f007-4d07-bf4c-2d72edc456c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.297229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c52b48-9e4f-4049-92b7-acd96dccfaa0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.297414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e5e3d7f-f007-4d07-bf4c-2d72edc456c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.298151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e3d7f-f007-4d07-bf4c-2d72edc456c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:30.298459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e3d7f-f007-4d07-bf4c-2d72edc456c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.300073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e5e3d7f-f007-4d07-bf4c-2d72edc456c1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.302122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52109eaf-f2ec-451e-9635-83636cadc5e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.302840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52109eaf-f2ec-451e-9635-83636cadc5e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.303129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52109eaf-f2ec-451e-9635-83636cadc5e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.303410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52109eaf-f2ec-451e-9635-83636cadc5e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.304163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52109eaf-f2ec-451e-9635-83636cadc5e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:30.304476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52109eaf-f2ec-451e-9635-83636cadc5e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.306190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52109eaf-f2ec-451e-9635-83636cadc5e3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.312202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 775/1563] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:30.313840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512eba3-70c1-4625-9d3c-1f328ffef5b3 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.314243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:30.315007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512eba3-70c1-4625-9d3c-1f328ffef5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.315452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7512eba3-70c1-4625-9d3c-1f328ffef5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.329659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 789/1564] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:30.329659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52109eaf-f2ec-451e-9635-83636cadc5e3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.337246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7512eba3-70c1-4625-9d3c-1f328ffef5b3 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:30.337678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52109eaf-f2ec-451e-9635-83636cadc5e3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.338618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7512eba3-70c1-4625-9d3c-1f328ffef5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:30.343073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15906413-4bb1-4b7b-a18d-dafc82e5df8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.343794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15906413-4bb1-4b7b-a18d-dafc82e5df8a admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.344076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15906413-4bb1-4b7b-a18d-dafc82e5df8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.344351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15906413-4bb1-4b7b-a18d-dafc82e5df8a admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.345072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15906413-4bb1-4b7b-a18d-dafc82e5df8a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:30.345381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15906413-4bb1-4b7b-a18d-dafc82e5df8a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.346897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 776/1565] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:30.347018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15906413-4bb1-4b7b-a18d-dafc82e5df8a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.350885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512eba3-70c1-4625-9d3c-1f328ffef5b3 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.360774 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 790/1566] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:30.361378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7512eba3-70c1-4625-9d3c-1f328ffef5b3 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.361626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15906413-4bb1-4b7b-a18d-dafc82e5df8a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.366163 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888848c-7983-4498-be16-196649cda571 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.366833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888848c-7983-4498-be16-196649cda571 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.367118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888848c-7983-4498-be16-196649cda571 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.367391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6888848c-7983-4498-be16-196649cda571 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.368246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888848c-7983-4498-be16-196649cda571 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.370265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6888848c-7983-4498-be16-196649cda571 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.372271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15906413-4bb1-4b7b-a18d-dafc82e5df8a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.376978 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6da8a2-5357-41fe-beff-f1c69a46b910 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.377893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6da8a2-5357-41fe-beff-f1c69a46b910 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.378179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6da8a2-5357-41fe-beff-f1c69a46b910 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.378456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6da8a2-5357-41fe-beff-f1c69a46b910 admin admin] PATH_INFO: `/v3/projects/05e0691b14804cc5b4fe120fc5dea871/users/beb901faa8e9410484f83e47e5bcfa8c/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.390923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 777/1567] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/users => generated 287 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:30.392193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888848c-7983-4498-be16-196649cda571 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.395813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6da8a2-5357-41fe-beff-f1c69a46b910 admin admin] RBAC: Authorizing `identity:create_grant(project_id=05e0691b14804cc5b4fe120fc5dea871, user_id=beb901faa8e9410484f83e47e5bcfa8c, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.396794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-9c6da8a2-5357-41fe-beff-f1c69a46b910 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:30.396794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 20 19:23:30.396794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: Jul 20 19:23:30.398818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6da8a2-5357-41fe-beff-f1c69a46b910 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.402962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6888848c-7983-4498-be16-196649cda571 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.408268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40115e2-6e6f-455f-8d57-084b9136d2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.408928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40115e2-6e6f-455f-8d57-084b9136d2ee admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.409217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40115e2-6e6f-455f-8d57-084b9136d2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.409497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40115e2-6e6f-455f-8d57-084b9136d2ee admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.410257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40115e2-6e6f-455f-8d57-084b9136d2ee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:30.410570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40115e2-6e6f-455f-8d57-084b9136d2ee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.412169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40115e2-6e6f-455f-8d57-084b9136d2ee admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.413504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 791/1568] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:30 2022] PUT /identity/v3/projects/05e0691b14804cc5b4fe120fc5dea871/users/beb901faa8e9410484f83e47e5bcfa8c/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:30.414543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6da8a2-5357-41fe-beff-f1c69a46b910 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.436169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 778/1569] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:30.436169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40115e2-6e6f-455f-8d57-084b9136d2ee admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.440481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40115e2-6e6f-455f-8d57-084b9136d2ee admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.485378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6da8a2-5357-41fe-beff-f1c69a46b910 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.490159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e92c62-e129-4195-985c-5d02502abf09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.490822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e92c62-e129-4195-985c-5d02502abf09 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.491107 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e92c62-e129-4195-985c-5d02502abf09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.491383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e92c62-e129-4195-985c-5d02502abf09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.492075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e92c62-e129-4195-985c-5d02502abf09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.493124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-a3e92c62-e129-4195-985c-5d02502abf09 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:23:30.493124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 20 19:23:30.493124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: Jul 20 19:23:30.494580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e92c62-e129-4195-985c-5d02502abf09 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.500913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92f738-a3e2-42b5-bcef-e60861fd1610 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.501621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92f738-a3e2-42b5-bcef-e60861fd1610 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.501906 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92f738-a3e2-42b5-bcef-e60861fd1610 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.502180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f92f738-a3e2-42b5-bcef-e60861fd1610 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.502867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92f738-a3e2-42b5-bcef-e60861fd1610 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.504772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f92f738-a3e2-42b5-bcef-e60861fd1610 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.523327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 792/1570] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/projects => generated 379 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:30.524241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e92c62-e129-4195-985c-5d02502abf09 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.542057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 779/1571] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/projects => generated 383 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:30.542057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92f738-a3e2-42b5-bcef-e60861fd1610 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.595582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e92c62-e129-4195-985c-5d02502abf09 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.600607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9315c-01df-4ce2-8080-d0423e4cdbc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.601281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9315c-01df-4ce2-8080-d0423e4cdbc8 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.601599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9315c-01df-4ce2-8080-d0423e4cdbc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.601884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac9315c-01df-4ce2-8080-d0423e4cdbc8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.602686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9315c-01df-4ce2-8080-d0423e4cdbc8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.604638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac9315c-01df-4ce2-8080-d0423e4cdbc8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.604882 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f92f738-a3e2-42b5-bcef-e60861fd1610 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.610626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ea054-46ac-4590-ae66-d9daf90f1667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.611448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ea054-46ac-4590-ae66-d9daf90f1667 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.611742 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ea054-46ac-4590-ae66-d9daf90f1667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.612020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-033ea054-46ac-4590-ae66-d9daf90f1667 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.612900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ea054-46ac-4590-ae66-d9daf90f1667 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:30.613253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ea054-46ac-4590-ae66-d9daf90f1667 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.615465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-033ea054-46ac-4590-ae66-d9daf90f1667 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.629926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 780/1572] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:30.630664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ea054-46ac-4590-ae66-d9daf90f1667 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.646313 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 793/1573] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/projects => generated 367 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:30.646878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-033ea054-46ac-4590-ae66-d9daf90f1667 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.647375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9315c-01df-4ce2-8080-d0423e4cdbc8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.659748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:30.686748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:30.714996 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43f051-f974-4cf4-a3c3-ccc01c3a881a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.715813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43f051-f974-4cf4-a3c3-ccc01c3a881a admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.716083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43f051-f974-4cf4-a3c3-ccc01c3a881a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.716361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43f051-f974-4cf4-a3c3-ccc01c3a881a admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.717066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43f051-f974-4cf4-a3c3-ccc01c3a881a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.719305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43f051-f974-4cf4-a3c3-ccc01c3a881a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.732192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac9315c-01df-4ce2-8080-d0423e4cdbc8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.739839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816af50e-914e-469d-8063-927bc7ea8f6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.740882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816af50e-914e-469d-8063-927bc7ea8f6c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.741458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816af50e-914e-469d-8063-927bc7ea8f6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.741875 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-816af50e-914e-469d-8063-927bc7ea8f6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.743096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816af50e-914e-469d-8063-927bc7ea8f6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.745967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-816af50e-914e-469d-8063-927bc7ea8f6c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.763752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 781/1574] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/projects => generated 389 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:30.763752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bd685-2064-4d74-aec5-10352257310e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.763752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bd685-2064-4d74-aec5-10352257310e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.763752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779bd685-2064-4d74-aec5-10352257310e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.765435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 794/1575] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:30 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:30.770703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816af50e-914e-469d-8063-927bc7ea8f6c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.783478 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-816af50e-914e-469d-8063-927bc7ea8f6c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.785034 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-779bd685-2064-4d74-aec5-10352257310e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:30.786298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-779bd685-2064-4d74-aec5-10352257310e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:30.868942 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 782/1576] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:30.870066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bd685-2064-4d74-aec5-10352257310e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.885317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779bd685-2064-4d74-aec5-10352257310e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:30.888486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7517895-6d1c-4a91-91da-2e9cee6ec37d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.889180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7517895-6d1c-4a91-91da-2e9cee6ec37d admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.889464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7517895-6d1c-4a91-91da-2e9cee6ec37d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.889813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7517895-6d1c-4a91-91da-2e9cee6ec37d admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.890484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7517895-6d1c-4a91-91da-2e9cee6ec37d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.891065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1069c-2b39-457f-8a83-68e301bf011c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:30.891744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1069c-2b39-457f-8a83-68e301bf011c admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:30.892030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1069c-2b39-457f-8a83-68e301bf011c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:30.892306 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfc1069c-2b39-457f-8a83-68e301bf011c admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:30.892450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7517895-6d1c-4a91-91da-2e9cee6ec37d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.893115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1069c-2b39-457f-8a83-68e301bf011c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:30.895080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfc1069c-2b39-457f-8a83-68e301bf011c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:30.923027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 783/1577] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/users => generated 290 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:30.923887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1069c-2b39-457f-8a83-68e301bf011c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.929387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 795/1578] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/projects => generated 393 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:30.929387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7517895-6d1c-4a91-91da-2e9cee6ec37d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:30.935380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfc1069c-2b39-457f-8a83-68e301bf011c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.006437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7517895-6d1c-4a91-91da-2e9cee6ec37d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.007336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1276e-4e0e-463e-9840-c5f7029c4b22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.008099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1276e-4e0e-463e-9840-c5f7029c4b22 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.008318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1276e-4e0e-463e-9840-c5f7029c4b22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.009151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f1276e-4e0e-463e-9840-c5f7029c4b22 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.010074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1276e-4e0e-463e-9840-c5f7029c4b22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.012034 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f1276e-4e0e-463e-9840-c5f7029c4b22 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.018821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd4dc4-a670-460d-b850-6517d70d33d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.019527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd4dc4-a670-460d-b850-6517d70d33d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.019801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd4dc4-a670-460d-b850-6517d70d33d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.020073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fd4dc4-a670-460d-b850-6517d70d33d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.020777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd4dc4-a670-460d-b850-6517d70d33d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.022721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fd4dc4-a670-460d-b850-6517d70d33d2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.038483 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 784/1579] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/users => generated 292 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:31.039928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1276e-4e0e-463e-9840-c5f7029c4b22 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.053110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f1276e-4e0e-463e-9840-c5f7029c4b22 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.053385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 796/1580] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:30 2022] POST /identity/v3/projects => generated 377 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:31.054110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd4dc4-a670-460d-b850-6517d70d33d2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.068269 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:31.107233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:31.125228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7a0fe-83d1-4196-8ef3-5ff9bf52e87a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.125984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7a0fe-83d1-4196-8ef3-5ff9bf52e87a admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.126271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7a0fe-83d1-4196-8ef3-5ff9bf52e87a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.126558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7a0fe-83d1-4196-8ef3-5ff9bf52e87a admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.127256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7a0fe-83d1-4196-8ef3-5ff9bf52e87a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.129231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7a0fe-83d1-4196-8ef3-5ff9bf52e87a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.131858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fd4dc4-a670-460d-b850-6517d70d33d2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.137394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd1663-dbce-4741-9fe7-4cfb7cc62dfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.138093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1663-dbce-4741-9fe7-4cfb7cc62dfc admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.138371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1663-dbce-4741-9fe7-4cfb7cc62dfc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.138942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65cd1663-dbce-4741-9fe7-4cfb7cc62dfc admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.139394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd1663-dbce-4741-9fe7-4cfb7cc62dfc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.141435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65cd1663-dbce-4741-9fe7-4cfb7cc62dfc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.149901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 785/1581] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:31 2022] POST /identity/v3/users => generated 284 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:31.150784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7a0fe-83d1-4196-8ef3-5ff9bf52e87a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.162256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7a0fe-83d1-4196-8ef3-5ff9bf52e87a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.165560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 797/1582] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:31 2022] POST /identity/v3/users => generated 295 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:31.166382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd1663-dbce-4741-9fe7-4cfb7cc62dfc admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.167958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-65cd1663-dbce-4741-9fe7-4cfb7cc62dfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:31.170420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4d5d2-57cd-4521-9aa2-6b899fa48856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.171120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4d5d2-57cd-4521-9aa2-6b899fa48856 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.171409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4d5d2-57cd-4521-9aa2-6b899fa48856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.171689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f4d5d2-57cd-4521-9aa2-6b899fa48856 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.172424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4d5d2-57cd-4521-9aa2-6b899fa48856 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.174119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f4d5d2-57cd-4521-9aa2-6b899fa48856 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.185749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 786/1583] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.186578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4d5d2-57cd-4521-9aa2-6b899fa48856 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.200025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f4d5d2-57cd-4521-9aa2-6b899fa48856 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.205379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccafb9b-8049-4350-a0c9-6921f28a9da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.206090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccafb9b-8049-4350-a0c9-6921f28a9da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.206379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccafb9b-8049-4350-a0c9-6921f28a9da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.206662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccafb9b-8049-4350-a0c9-6921f28a9da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.207397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccafb9b-8049-4350-a0c9-6921f28a9da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.209019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccafb9b-8049-4350-a0c9-6921f28a9da8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.220290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 787/1584] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.221192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccafb9b-8049-4350-a0c9-6921f28a9da8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.232712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccafb9b-8049-4350-a0c9-6921f28a9da8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.237431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07124139-e5d3-4777-ae7a-ebc1e3d2189e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.238281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07124139-e5d3-4777-ae7a-ebc1e3d2189e admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.238640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07124139-e5d3-4777-ae7a-ebc1e3d2189e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.238908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07124139-e5d3-4777-ae7a-ebc1e3d2189e admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.240267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07124139-e5d3-4777-ae7a-ebc1e3d2189e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.241572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07124139-e5d3-4777-ae7a-ebc1e3d2189e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.241976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65cd1663-dbce-4741-9fe7-4cfb7cc62dfc admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.246694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083f36f-be60-46bf-af91-f70fb30ae707 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.247365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083f36f-be60-46bf-af91-f70fb30ae707 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.247647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083f36f-be60-46bf-af91-f70fb30ae707 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.247926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1083f36f-be60-46bf-af91-f70fb30ae707 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.263281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083f36f-be60-46bf-af91-f70fb30ae707 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.264774 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1083f36f-be60-46bf-af91-f70fb30ae707 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.267751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 788/1585] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:31 2022] POST /identity/v3/users => generated 297 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:31.269221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07124139-e5d3-4777-ae7a-ebc1e3d2189e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.279517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 798/1586] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:31.280807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083f36f-be60-46bf-af91-f70fb30ae707 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.293059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07124139-e5d3-4777-ae7a-ebc1e3d2189e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.293417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1083f36f-be60-46bf-af91-f70fb30ae707 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.298282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608f71d-61ee-4de1-aa29-7a2c99f28921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.299308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608f71d-61ee-4de1-aa29-7a2c99f28921 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.299939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608f71d-61ee-4de1-aa29-7a2c99f28921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.299939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d608f71d-61ee-4de1-aa29-7a2c99f28921 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.300251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef8bbc-6e3a-44f2-9213-32c35452995e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.300474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608f71d-61ee-4de1-aa29-7a2c99f28921 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.301373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef8bbc-6e3a-44f2-9213-32c35452995e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.301609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef8bbc-6e3a-44f2-9213-32c35452995e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.302167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fef8bbc-6e3a-44f2-9213-32c35452995e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.302771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d608f71d-61ee-4de1-aa29-7a2c99f28921 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.303064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef8bbc-6e3a-44f2-9213-32c35452995e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.305302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fef8bbc-6e3a-44f2-9213-32c35452995e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.327281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 789/1587] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.328924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef8bbc-6e3a-44f2-9213-32c35452995e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.339364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 799/1588] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:31 2022] POST /identity/v3/users => generated 289 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:31.341000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608f71d-61ee-4de1-aa29-7a2c99f28921 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.344629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fef8bbc-6e3a-44f2-9213-32c35452995e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.352748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3565b-d8d0-45cc-957c-798fafdaf917 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.354294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3565b-d8d0-45cc-957c-798fafdaf917 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.354654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3565b-d8d0-45cc-957c-798fafdaf917 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.354999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af3565b-d8d0-45cc-957c-798fafdaf917 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.356169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3565b-d8d0-45cc-957c-798fafdaf917 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.357019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d608f71d-61ee-4de1-aa29-7a2c99f28921 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.358617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af3565b-d8d0-45cc-957c-798fafdaf917 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.362902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea45ad3-bbeb-4dea-88a4-f3bc8e07b208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.363625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea45ad3-bbeb-4dea-88a4-f3bc8e07b208 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.363904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea45ad3-bbeb-4dea-88a4-f3bc8e07b208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.364183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea45ad3-bbeb-4dea-88a4-f3bc8e07b208 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.364978 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea45ad3-bbeb-4dea-88a4-f3bc8e07b208 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.366760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea45ad3-bbeb-4dea-88a4-f3bc8e07b208 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.373991 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 790/1589] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.378674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3565b-d8d0-45cc-957c-798fafdaf917 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.381881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 800/1590] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.382819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea45ad3-bbeb-4dea-88a4-f3bc8e07b208 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.391168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af3565b-d8d0-45cc-957c-798fafdaf917 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.393831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea45ad3-bbeb-4dea-88a4-f3bc8e07b208 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.397502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549f95-3df6-4ecb-9cf7-a27f7123865e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.398773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549f95-3df6-4ecb-9cf7-a27f7123865e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.400652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fb267-6b1d-4be1-ad91-47ae87e43d12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.400818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549f95-3df6-4ecb-9cf7-a27f7123865e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.400818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a549f95-3df6-4ecb-9cf7-a27f7123865e admin admin] PATH_INFO: `/v3/projects/7fedcb34f863465890f3b64591063546/users/d3f8f0bd2cbb4ae9b51990b6efec2b92/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.401224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fb267-6b1d-4be1-ad91-47ae87e43d12 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.401224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fb267-6b1d-4be1-ad91-47ae87e43d12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.401224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51fb267-6b1d-4be1-ad91-47ae87e43d12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.401643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fb267-6b1d-4be1-ad91-47ae87e43d12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.403311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51fb267-6b1d-4be1-ad91-47ae87e43d12 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.415729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 801/1591] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.416683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fb267-6b1d-4be1-ad91-47ae87e43d12 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.424316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a549f95-3df6-4ecb-9cf7-a27f7123865e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7fedcb34f863465890f3b64591063546, user_id=d3f8f0bd2cbb4ae9b51990b6efec2b92, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.427918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a549f95-3df6-4ecb-9cf7-a27f7123865e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.432005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51fb267-6b1d-4be1-ad91-47ae87e43d12 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.438144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96efec-1b78-4ece-90e8-7613ac7f2d55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.438828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96efec-1b78-4ece-90e8-7613ac7f2d55 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.439097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96efec-1b78-4ece-90e8-7613ac7f2d55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.439372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a96efec-1b78-4ece-90e8-7613ac7f2d55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.440106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96efec-1b78-4ece-90e8-7613ac7f2d55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.442018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a96efec-1b78-4ece-90e8-7613ac7f2d55 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.453733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 802/1593] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.454487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96efec-1b78-4ece-90e8-7613ac7f2d55 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.461673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 791/1593] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:31 2022] PUT /identity/v3/projects/7fedcb34f863465890f3b64591063546/users/d3f8f0bd2cbb4ae9b51990b6efec2b92/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:31.463511 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549f95-3df6-4ecb-9cf7-a27f7123865e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.467910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a96efec-1b78-4ece-90e8-7613ac7f2d55 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.538120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f973cd-96dc-49ff-96f8-fd8324f38ae3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.538832 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f973cd-96dc-49ff-96f8-fd8324f38ae3 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.539088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f973cd-96dc-49ff-96f8-fd8324f38ae3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.539371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f973cd-96dc-49ff-96f8-fd8324f38ae3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.540853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f973cd-96dc-49ff-96f8-fd8324f38ae3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.542736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f973cd-96dc-49ff-96f8-fd8324f38ae3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.553583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a549f95-3df6-4ecb-9cf7-a27f7123865e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.554435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 803/1594] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.555306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f973cd-96dc-49ff-96f8-fd8324f38ae3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.563166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cfa6a-d91d-417a-96ca-6b6a6786c3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.563863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cfa6a-d91d-417a-96ca-6b6a6786c3a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.564151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cfa6a-d91d-417a-96ca-6b6a6786c3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.564457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1cfa6a-d91d-417a-96ca-6b6a6786c3a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.565193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1cfa6a-d91d-417a-96ca-6b6a6786c3a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.566978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1cfa6a-d91d-417a-96ca-6b6a6786c3a3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.567258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f973cd-96dc-49ff-96f8-fd8324f38ae3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.575456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cf9f1-e8de-421a-a702-94160a863252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.576268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cf9f1-e8de-421a-a702-94160a863252 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.576622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cf9f1-e8de-421a-a702-94160a863252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.576971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125cf9f1-e8de-421a-a702-94160a863252 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.577843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cf9f1-e8de-421a-a702-94160a863252 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.579175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 792/1595] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.580114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cfa6a-d91d-417a-96ca-6b6a6786c3a3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.582137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125cf9f1-e8de-421a-a702-94160a863252 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.598302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1cfa6a-d91d-417a-96ca-6b6a6786c3a3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.603504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 804/1596] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.604815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cf9f1-e8de-421a-a702-94160a863252 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.608362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbcb12-b33d-4486-8d57-3f7aab1cd578 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.609311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbcb12-b33d-4486-8d57-3f7aab1cd578 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.609799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbcb12-b33d-4486-8d57-3f7aab1cd578 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.610250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dbcb12-b33d-4486-8d57-3f7aab1cd578 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.611832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbcb12-b33d-4486-8d57-3f7aab1cd578 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.613467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:31.616231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dbcb12-b33d-4486-8d57-3f7aab1cd578 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.619400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125cf9f1-e8de-421a-a702-94160a863252 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.626005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7b8a0-cdf6-4fa5-99b6-210f938a4cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.627390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7b8a0-cdf6-4fa5-99b6-210f938a4cc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.628016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7b8a0-cdf6-4fa5-99b6-210f938a4cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.628804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe7b8a0-cdf6-4fa5-99b6-210f938a4cc2 admin admin] PATH_INFO: `/v3/projects/0dd360813c734ddfa4e36fd90acfcb1c/users/28551e3575b24089b0043692f65a8ffd/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.636561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 793/1597] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.637896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbcb12-b33d-4486-8d57-3f7aab1cd578 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.645675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:31.653869 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dbcb12-b33d-4486-8d57-3f7aab1cd578 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.660564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29f042c4-8229-4194-a16a-926bde15325e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.661362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f042c4-8229-4194-a16a-926bde15325e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.662343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f042c4-8229-4194-a16a-926bde15325e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.663245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f042c4-8229-4194-a16a-926bde15325e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.664826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7b8a0-cdf6-4fa5-99b6-210f938a4cc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dd360813c734ddfa4e36fd90acfcb1c, user_id=28551e3575b24089b0043692f65a8ffd, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.665950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f042c4-8229-4194-a16a-926bde15325e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.670352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29f042c4-8229-4194-a16a-926bde15325e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.671937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe7b8a0-cdf6-4fa5-99b6-210f938a4cc2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.685056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 794/1598] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.686993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b2642-12e4-470b-8822-a1b9720c9cf1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.687291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b2642-12e4-470b-8822-a1b9720c9cf1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.687671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163b2642-12e4-470b-8822-a1b9720c9cf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.702672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 805/1599] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:31 2022] PUT /identity/v3/projects/0dd360813c734ddfa4e36fd90acfcb1c/users/28551e3575b24089b0043692f65a8ffd/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:31.703706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7b8a0-cdf6-4fa5-99b6-210f938a4cc2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.722190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-163b2642-12e4-470b-8822-a1b9720c9cf1 None None] MFA Rules not processed for user `d3f8f0bd2cbb4ae9b51990b6efec2b92`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:31.723440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-163b2642-12e4-470b-8822-a1b9720c9cf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:31.793258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe7b8a0-cdf6-4fa5-99b6-210f938a4cc2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.802150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f19550-4149-4763-8fe1-60c3dc42719e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.803285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f19550-4149-4763-8fe1-60c3dc42719e admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.803670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f19550-4149-4763-8fe1-60c3dc42719e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.804046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f19550-4149-4763-8fe1-60c3dc42719e admin admin] PATH_INFO: `/v3/projects/f97d002af3ee402090e62737cf1e14b9/users/47478871f08e4c58be083739eb1ee56b/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.804632 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 795/1600] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:31.817491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b2642-12e4-470b-8822-a1b9720c9cf1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.829291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f19550-4149-4763-8fe1-60c3dc42719e admin admin] RBAC: Authorizing `identity:create_grant(project_id=f97d002af3ee402090e62737cf1e14b9, user_id=47478871f08e4c58be083739eb1ee56b, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.832587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f19550-4149-4763-8fe1-60c3dc42719e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.838632 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163b2642-12e4-470b-8822-a1b9720c9cf1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.847871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152a789-962c-4a91-85b8-8240ce2b57e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.849038 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152a789-962c-4a91-85b8-8240ce2b57e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.850100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152a789-962c-4a91-85b8-8240ce2b57e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.851714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a152a789-962c-4a91-85b8-8240ce2b57e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.852927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 806/1601] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:31 2022] PUT /identity/v3/projects/f97d002af3ee402090e62737cf1e14b9/users/47478871f08e4c58be083739eb1ee56b/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:31.853959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152a789-962c-4a91-85b8-8240ce2b57e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.854468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f19550-4149-4763-8fe1-60c3dc42719e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.856434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a152a789-962c-4a91-85b8-8240ce2b57e6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.871673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 796/1602] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:31 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:31.872313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152a789-962c-4a91-85b8-8240ce2b57e6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:31.883388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a152a789-962c-4a91-85b8-8240ce2b57e6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.941152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f19550-4149-4763-8fe1-60c3dc42719e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:31.948127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a770352-70ca-4e13-85d7-af78a6ebdc9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.949283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a770352-70ca-4e13-85d7-af78a6ebdc9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.949680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a770352-70ca-4e13-85d7-af78a6ebdc9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.950184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a770352-70ca-4e13-85d7-af78a6ebdc9c admin admin] PATH_INFO: `/v3/projects/53495064e7074f4a9d5fc05907272405/users/3c1e5bfe31b348d1891206de7e4706eb/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.958674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce449eb2-50f5-4f0e-b627-3fbee9dfbb50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:31.959585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce449eb2-50f5-4f0e-b627-3fbee9dfbb50 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:31.959872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce449eb2-50f5-4f0e-b627-3fbee9dfbb50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:31.960152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce449eb2-50f5-4f0e-b627-3fbee9dfbb50 admin admin] PATH_INFO: `/v3/projects/e6e0bccb45a24af3918f7db2c60de33d/users/343e480dd5ee4389ba430129cd7daaa4/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:31.967149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a770352-70ca-4e13-85d7-af78a6ebdc9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=53495064e7074f4a9d5fc05907272405, user_id=3c1e5bfe31b348d1891206de7e4706eb, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.970700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a770352-70ca-4e13-85d7-af78a6ebdc9c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:31.978510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce449eb2-50f5-4f0e-b627-3fbee9dfbb50 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e6e0bccb45a24af3918f7db2c60de33d, user_id=343e480dd5ee4389ba430129cd7daaa4, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:31.981636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce449eb2-50f5-4f0e-b627-3fbee9dfbb50 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:32.000041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 807/1603] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:31 2022] PUT /identity/v3/projects/53495064e7074f4a9d5fc05907272405/users/3c1e5bfe31b348d1891206de7e4706eb/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:32.001090 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a770352-70ca-4e13-85d7-af78a6ebdc9c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:32.017016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 797/1604] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:31 2022] PUT /identity/v3/projects/e6e0bccb45a24af3918f7db2c60de33d/users/343e480dd5ee4389ba430129cd7daaa4/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:32.022312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfc1b0-92f5-4f46-a075-9dd129006a56 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.022672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfc1b0-92f5-4f46-a075-9dd129006a56 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.023023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bbfc1b0-92f5-4f46-a075-9dd129006a56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.058115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6bbfc1b0-92f5-4f46-a075-9dd129006a56 None None] MFA Rules not processed for user `28551e3575b24089b0043692f65a8ffd`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:32.060130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6bbfc1b0-92f5-4f46-a075-9dd129006a56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:32.065901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:32.089953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a770352-70ca-4e13-85d7-af78a6ebdc9c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:32.115271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:32.143472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 798/1605] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:32.145209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f91e05-91b7-4a73-9a2b-b71facf8bd9e None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.145720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f91e05-91b7-4a73-9a2b-b71facf8bd9e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.146040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f91e05-91b7-4a73-9a2b-b71facf8bd9e None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.147686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 799/1606] 10.222.0.69 () {54 vars in 962 bytes} [Wed Jul 20 19:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:32.149258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252432c1-5d7c-42b4-8491-0e22a9132b35 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.149258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252432c1-5d7c-42b4-8491-0e22a9132b35 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.149618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252432c1-5d7c-42b4-8491-0e22a9132b35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.178524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-252432c1-5d7c-42b4-8491-0e22a9132b35 None None] MFA Rules not processed for user `47478871f08e4c58be083739eb1ee56b`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:32.179978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-252432c1-5d7c-42b4-8491-0e22a9132b35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:32.182099 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca7b7f-f84e-4f52-b816-81b52a98e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:32.183214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca7b7f-f84e-4f52-b816-81b52a98e3af admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.183506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca7b7f-f84e-4f52-b816-81b52a98e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.183784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbca7b7f-f84e-4f52-b816-81b52a98e3af admin admin] PATH_INFO: `/v3/projects/8821facab6ed48588d9d484b5e3d98ba/users/9192f2842ca044f08e5f95f98f2e1b88/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.202967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca7b7f-f84e-4f52-b816-81b52a98e3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=8821facab6ed48588d9d484b5e3d98ba, user_id=9192f2842ca044f08e5f95f98f2e1b88, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:32.206023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbca7b7f-f84e-4f52-b816-81b52a98e3af admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:32.225370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 808/1607] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:32 2022] PUT /identity/v3/projects/8821facab6ed48588d9d484b5e3d98ba/users/9192f2842ca044f08e5f95f98f2e1b88/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:32.226241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca7b7f-f84e-4f52-b816-81b52a98e3af admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:32.271953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 800/1608] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:32.271953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db64ea9-82be-4e93-9bb1-4abbf005ad1c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.271953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db64ea9-82be-4e93-9bb1-4abbf005ad1c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.271953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db64ea9-82be-4e93-9bb1-4abbf005ad1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.307247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4db64ea9-82be-4e93-9bb1-4abbf005ad1c None None] MFA Rules not processed for user `3c1e5bfe31b348d1891206de7e4706eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:32.308866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4db64ea9-82be-4e93-9bb1-4abbf005ad1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:32.330866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbca7b7f-f84e-4f52-b816-81b52a98e3af admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:32.340002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39303895-7102-45f5-91f4-b24172c636f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:32.341932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39303895-7102-45f5-91f4-b24172c636f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.342380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39303895-7102-45f5-91f4-b24172c636f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.343080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39303895-7102-45f5-91f4-b24172c636f2 admin admin] PATH_INFO: `/v3/projects/d0ab510f529d4442bd0e379056c4c3e5/users/4d08da0ef69e4cf2abc6c18bbbc51a2f/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.367027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39303895-7102-45f5-91f4-b24172c636f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0ab510f529d4442bd0e379056c4c3e5, user_id=4d08da0ef69e4cf2abc6c18bbbc51a2f, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:32.371410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39303895-7102-45f5-91f4-b24172c636f2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:32.393588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 809/1609] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:32 2022] PUT /identity/v3/projects/d0ab510f529d4442bd0e379056c4c3e5/users/4d08da0ef69e4cf2abc6c18bbbc51a2f/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:32.395485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca37198-5c5d-45d6-9b6c-a1978562aa7e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.396398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca37198-5c5d-45d6-9b6c-a1978562aa7e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.396915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca37198-5c5d-45d6-9b6c-a1978562aa7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.410998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 801/1610] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:32.417150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db64ea9-82be-4e93-9bb1-4abbf005ad1c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:32.430249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db64ea9-82be-4e93-9bb1-4abbf005ad1c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:32.439290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-fca37198-5c5d-45d6-9b6c-a1978562aa7e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:32.442120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fca37198-5c5d-45d6-9b6c-a1978562aa7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:32.503689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ab916-d4ab-4858-aad6-fda603b25f44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:32.504424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ab916-d4ab-4858-aad6-fda603b25f44 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.504716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ab916-d4ab-4858-aad6-fda603b25f44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.505039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ab916-d4ab-4858-aad6-fda603b25f44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.505937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ab916-d4ab-4858-aad6-fda603b25f44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:32.508051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ab916-d4ab-4858-aad6-fda603b25f44 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:32.513141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:32.522969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:32.539453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 810/1611] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:32.546026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33378284-a9bb-4268-9d53-a90b79d9d5bb None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.546325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33378284-a9bb-4268-9d53-a90b79d9d5bb None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.546615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33378284-a9bb-4268-9d53-a90b79d9d5bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.550920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 802/1612] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/projects => generated 381 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:32.553000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0705db-ef76-4ee7-a692-cf427e967517 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.553142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0705db-ef76-4ee7-a692-cf427e967517 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.553779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0705db-ef76-4ee7-a692-cf427e967517 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.574791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fb0705db-ef76-4ee7-a692-cf427e967517 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:32.576156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fb0705db-ef76-4ee7-a692-cf427e967517 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:32.607581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-33378284-a9bb-4268-9d53-a90b79d9d5bb None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:32.609469 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-33378284-a9bb-4268-9d53-a90b79d9d5bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:32.666383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 803/1613] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:32.668050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66848b5c-feb4-41f2-a38f-371ce50433eb None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.668347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66848b5c-feb4-41f2-a38f-371ce50433eb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.668793 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66848b5c-feb4-41f2-a38f-371ce50433eb None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.671489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 804/1614] 10.222.0.69 () {54 vars in 962 bytes} [Wed Jul 20 19:23:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:32.671943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfb3a0-3f73-45f6-bf42-c7d2f54ddbe7 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.672070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfb3a0-3f73-45f6-bf42-c7d2f54ddbe7 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.672305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fedfb3a0-3f73-45f6-bf42-c7d2f54ddbe7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.702758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fedfb3a0-3f73-45f6-bf42-c7d2f54ddbe7 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:32.704515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fedfb3a0-3f73-45f6-bf42-c7d2f54ddbe7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:32.709066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 811/1615] 10.222.0.69 () {58 vars in 1059 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:32.710720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33378284-a9bb-4268-9d53-a90b79d9d5bb None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:32.716931 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 805/1616] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:32.718202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfb3a0-3f73-45f6-bf42-c7d2f54ddbe7 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:32.726976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33378284-a9bb-4268-9d53-a90b79d9d5bb None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:32.733354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da3495-edeb-48c7-ab3a-b6f166dfce0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:32.734352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da3495-edeb-48c7-ab3a-b6f166dfce0d admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.734719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da3495-edeb-48c7-ab3a-b6f166dfce0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.735094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30da3495-edeb-48c7-ab3a-b6f166dfce0d admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.736457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da3495-edeb-48c7-ab3a-b6f166dfce0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:32.738679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30da3495-edeb-48c7-ab3a-b6f166dfce0d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:32.780275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 812/1617] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/projects => generated 369 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:32.781166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da3495-edeb-48c7-ab3a-b6f166dfce0d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:32.824783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fedfb3a0-3f73-45f6-bf42-c7d2f54ddbe7 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:32.898591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30da3495-edeb-48c7-ab3a-b6f166dfce0d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:32.904267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75ef7f-c8a0-4a24-aa01-bc4c7e30656c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:32.904949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75ef7f-c8a0-4a24-aa01-bc4c7e30656c admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.905231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75ef7f-c8a0-4a24-aa01-bc4c7e30656c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.905563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb75ef7f-c8a0-4a24-aa01-bc4c7e30656c admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.906363 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75ef7f-c8a0-4a24-aa01-bc4c7e30656c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:32.908305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb75ef7f-c8a0-4a24-aa01-bc4c7e30656c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:32.923430 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4733e857-a637-49d7-8c7f-b8b713baced2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:32.924111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733e857-a637-49d7-8c7f-b8b713baced2 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.924390 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733e857-a637-49d7-8c7f-b8b713baced2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.924795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4733e857-a637-49d7-8c7f-b8b713baced2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.938710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 813/1618] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/users => generated 297 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:32.939966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75ef7f-c8a0-4a24-aa01-bc4c7e30656c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:32.942569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733e857-a637-49d7-8c7f-b8b713baced2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:32.944897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4733e857-a637-49d7-8c7f-b8b713baced2 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:32.951571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb75ef7f-c8a0-4a24-aa01-bc4c7e30656c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:32.956641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091d190-4f6f-4327-bb43-f337df931593 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:32.958337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091d190-4f6f-4327-bb43-f337df931593 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.958850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091d190-4f6f-4327-bb43-f337df931593 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.959340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8091d190-4f6f-4327-bb43-f337df931593 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.970384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 806/1619] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:32.971903 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475b42a-53c2-41e6-b101-00c0041c2798 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:32.972197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475b42a-53c2-41e6-b101-00c0041c2798 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:32.972477 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5475b42a-53c2-41e6-b101-00c0041c2798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:32.973084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091d190-4f6f-4327-bb43-f337df931593 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:32.975023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8091d190-4f6f-4327-bb43-f337df931593 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:32.988537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 814/1620] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:32.989963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091d190-4f6f-4327-bb43-f337df931593 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.015219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5475b42a-53c2-41e6-b101-00c0041c2798 None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:33.017833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5475b42a-53c2-41e6-b101-00c0041c2798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:33.047623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:33.082734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:33.114405 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8091d190-4f6f-4327-bb43-f337df931593 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:33.131379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15200bb-a0af-4554-b665-c5026f2f0329 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:33.132702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15200bb-a0af-4554-b665-c5026f2f0329 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.133213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15200bb-a0af-4554-b665-c5026f2f0329 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.134458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f15200bb-a0af-4554-b665-c5026f2f0329 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.152841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 807/1621] 10.222.0.69 () {58 vars in 1059 bytes} [Wed Jul 20 19:23:32 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:33.154725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475b42a-53c2-41e6-b101-00c0041c2798 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.178270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5475b42a-53c2-41e6-b101-00c0041c2798 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:33.189355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d236fcf-fdb2-416d-9ebf-c056c4657806 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:33.190075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d236fcf-fdb2-416d-9ebf-c056c4657806 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.190365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d236fcf-fdb2-416d-9ebf-c056c4657806 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.190648 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d236fcf-fdb2-416d-9ebf-c056c4657806 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.263376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d236fcf-fdb2-416d-9ebf-c056c4657806 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:33.263376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d236fcf-fdb2-416d-9ebf-c056c4657806 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:33.263376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 808/1622] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:33.268123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15200bb-a0af-4554-b665-c5026f2f0329 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:33.268345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d236fcf-fdb2-416d-9ebf-c056c4657806 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.268433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f15200bb-a0af-4554-b665-c5026f2f0329 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:33.268526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d236fcf-fdb2-416d-9ebf-c056c4657806 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:33.268647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 815/1623] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:32 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 277 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:33.268647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15200bb-a0af-4554-b665-c5026f2f0329 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.277277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebc663-0d68-4a7d-baa7-8da66c832b9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:33.278001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebc663-0d68-4a7d-baa7-8da66c832b9e admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.278287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebc663-0d68-4a7d-baa7-8da66c832b9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.278569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddebc663-0d68-4a7d-baa7-8da66c832b9e admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.280450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebc663-0d68-4a7d-baa7-8da66c832b9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:33.284842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddebc663-0d68-4a7d-baa7-8da66c832b9e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:33.291756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f15200bb-a0af-4554-b665-c5026f2f0329 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:33.304833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409dc18-f53b-44ba-a55c-9426113ddfe3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:33.305380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409dc18-f53b-44ba-a55c-9426113ddfe3 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.305816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409dc18-f53b-44ba-a55c-9426113ddfe3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.306095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409dc18-f53b-44ba-a55c-9426113ddfe3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.308804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409dc18-f53b-44ba-a55c-9426113ddfe3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:33.313389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409dc18-f53b-44ba-a55c-9426113ddfe3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:33.323794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 816/1624] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:33.325023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409dc18-f53b-44ba-a55c-9426113ddfe3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.333582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 809/1625] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:33 2022] POST /identity/v3/users => generated 291 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:33.333582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebc663-0d68-4a7d-baa7-8da66c832b9e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.362833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddebc663-0d68-4a7d-baa7-8da66c832b9e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:33.363263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409dc18-f53b-44ba-a55c-9426113ddfe3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:33.377104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e69f3f-431f-47a1-a241-009439b9c473 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:33.377933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69f3f-431f-47a1-a241-009439b9c473 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.378232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69f3f-431f-47a1-a241-009439b9c473 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.378516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e69f3f-431f-47a1-a241-009439b9c473 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.381760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d527e5-4247-4542-b209-0df8dad8cbf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:33.382941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d527e5-4247-4542-b209-0df8dad8cbf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.383576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d527e5-4247-4542-b209-0df8dad8cbf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.384207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d527e5-4247-4542-b209-0df8dad8cbf5 admin admin] PATH_INFO: `/v3/projects/ff8cb4a4fb48431db60b8b5e89c51ea1/users/72d02479b2b94506be98d917a3964880/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.417254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d527e5-4247-4542-b209-0df8dad8cbf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8cb4a4fb48431db60b8b5e89c51ea1, user_id=72d02479b2b94506be98d917a3964880, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:33.421370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68d527e5-4247-4542-b209-0df8dad8cbf5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:33.460464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 810/1626] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:33 2022] PUT /identity/v3/projects/ff8cb4a4fb48431db60b8b5e89c51ea1/users/72d02479b2b94506be98d917a3964880/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:33.466167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e69f3f-431f-47a1-a241-009439b9c473 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:33.467568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d527e5-4247-4542-b209-0df8dad8cbf5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.468517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01e69f3f-431f-47a1-a241-009439b9c473 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:33.490335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 817/1627] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:33 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:33.491275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e69f3f-431f-47a1-a241-009439b9c473 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.515502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e69f3f-431f-47a1-a241-009439b9c473 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:33.662288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d527e5-4247-4542-b209-0df8dad8cbf5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:33.678646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ace74-6307-4f5c-9697-73e1014a91e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:33.679352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ace74-6307-4f5c-9697-73e1014a91e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.679637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ace74-6307-4f5c-9697-73e1014a91e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.679915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37ace74-6307-4f5c-9697-73e1014a91e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.690596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ace74-6307-4f5c-9697-73e1014a91e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:33.695390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:33.698360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37ace74-6307-4f5c-9697-73e1014a91e9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:33.719895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43049fb2-0546-40eb-85cc-5a9dcf096427 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:33.721156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049fb2-0546-40eb-85cc-5a9dcf096427 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.721452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049fb2-0546-40eb-85cc-5a9dcf096427 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.721771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43049fb2-0546-40eb-85cc-5a9dcf096427 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.722567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43049fb2-0546-40eb-85cc-5a9dcf096427 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:33.725144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43049fb2-0546-40eb-85cc-5a9dcf096427 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:33.728822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 811/1628] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:33.729940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ace74-6307-4f5c-9697-73e1014a91e9 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.740884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 818/1629] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:33.745775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43049fb2-0546-40eb-85cc-5a9dcf096427 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.755166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37ace74-6307-4f5c-9697-73e1014a91e9 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:33.762073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43049fb2-0546-40eb-85cc-5a9dcf096427 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:33.770391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275d0b4-bdb9-47c9-9c50-b0f515e66bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:33.771435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275d0b4-bdb9-47c9-9c50-b0f515e66bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.771755 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275d0b4-bdb9-47c9-9c50-b0f515e66bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.772043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4275d0b4-bdb9-47c9-9c50-b0f515e66bd5 admin admin] PATH_INFO: `/v3/projects/b75b8a69938b47059767ba2edccff194/users/50484199e9e54240b01e7ecda02e9fa2/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.780678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee6df8a-03de-4855-ae3d-d95aa765ce0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:33.783044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6df8a-03de-4855-ae3d-d95aa765ce0f admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.784802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6df8a-03de-4855-ae3d-d95aa765ce0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.785917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee6df8a-03de-4855-ae3d-d95aa765ce0f admin admin] PATH_INFO: `/v3/projects/ff8cb4a4fb48431db60b8b5e89c51ea1/users/72d02479b2b94506be98d917a3964880/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.792284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275d0b4-bdb9-47c9-9c50-b0f515e66bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75b8a69938b47059767ba2edccff194, user_id=50484199e9e54240b01e7ecda02e9fa2, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:33.793340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6df8a-03de-4855-ae3d-d95aa765ce0f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff8cb4a4fb48431db60b8b5e89c51ea1, user_id=72d02479b2b94506be98d917a3964880, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:33.795031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4275d0b4-bdb9-47c9-9c50-b0f515e66bd5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:33.804352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee6df8a-03de-4855-ae3d-d95aa765ce0f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:33.823233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 812/1630] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:33 2022] PUT /identity/v3/projects/b75b8a69938b47059767ba2edccff194/users/50484199e9e54240b01e7ecda02e9fa2/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:23:33.827224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275d0b4-bdb9-47c9-9c50-b0f515e66bd5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:33.836561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 819/1631] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:33 2022] PUT /identity/v3/projects/ff8cb4a4fb48431db60b8b5e89c51ea1/users/72d02479b2b94506be98d917a3964880/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:33.847438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4e5df-d925-4111-8121-02f5aa653688 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:33.847975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4e5df-d925-4111-8121-02f5aa653688 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:33.848440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c4e5df-d925-4111-8121-02f5aa653688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:33.918483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-33c4e5df-d925-4111-8121-02f5aa653688 None None] MFA Rules not processed for user `72d02479b2b94506be98d917a3964880`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:33.920976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-33c4e5df-d925-4111-8121-02f5aa653688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:33.975714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:34.006090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4275d0b4-bdb9-47c9-9c50-b0f515e66bd5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:34.102299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 820/1632] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:23:34.104495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7298cef-0b2f-46b5-8dd4-b2fe97b91cd5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:34.104800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7298cef-0b2f-46b5-8dd4-b2fe97b91cd5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:34.105086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7298cef-0b2f-46b5-8dd4-b2fe97b91cd5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:34.106520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 821/1633] 10.222.0.69 () {54 vars in 912 bytes} [Wed Jul 20 19:23:34 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:34.107691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037e4b3-7e4c-4928-b2fe-18734fb9f86c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:34.108046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037e4b3-7e4c-4928-b2fe-18734fb9f86c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:34.108328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e037e4b3-7e4c-4928-b2fe-18734fb9f86c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:34.109768 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 822/1634] 10.222.0.69 () {54 vars in 912 bytes} [Wed Jul 20 19:23:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:34.116546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdd8fe-37b9-4994-bde6-1213d8252308 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:34.116835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdd8fe-37b9-4994-bde6-1213d8252308 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:34.117120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcdd8fe-37b9-4994-bde6-1213d8252308 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:34.142799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7fcdd8fe-37b9-4994-bde6-1213d8252308 None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:34.144346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7fcdd8fe-37b9-4994-bde6-1213d8252308 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:34.162373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7326fb-7b6d-4c1b-9db8-9d44b0e63867 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:34.164000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7326fb-7b6d-4c1b-9db8-9d44b0e63867 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:34.165763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7326fb-7b6d-4c1b-9db8-9d44b0e63867 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:34.166750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7326fb-7b6d-4c1b-9db8-9d44b0e63867 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:34.168187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7326fb-7b6d-4c1b-9db8-9d44b0e63867 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:34.171846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7326fb-7b6d-4c1b-9db8-9d44b0e63867 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:34.203998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 813/1635] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:33 2022] GET /identity/v3/roles => generated 1420 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:34.210717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c9e6-c1f7-4170-b4a5-d77b80ddf3ac None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:34.211537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c9e6-c1f7-4170-b4a5-d77b80ddf3ac None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:34.211805 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b961c9e6-c1f7-4170-b4a5-d77b80ddf3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:34.277258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b961c9e6-c1f7-4170-b4a5-d77b80ddf3ac None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:34.283043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b961c9e6-c1f7-4170-b4a5-d77b80ddf3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:34.295881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 823/1636] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 19:23:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:34.298118 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdd8fe-37b9-4994-bde6-1213d8252308 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:34.325135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 814/1637] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 19:23:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:34.325973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961c9e6-c1f7-4170-b4a5-d77b80ddf3ac None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:34.347450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b961c9e6-c1f7-4170-b4a5-d77b80ddf3ac None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:34.355297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9819b4-de71-4319-922b-526c1c05f7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:34.356915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9819b4-de71-4319-922b-526c1c05f7a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:34.357291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9819b4-de71-4319-922b-526c1c05f7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:34.357677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c9819b4-de71-4319-922b-526c1c05f7a5 admin admin] PATH_INFO: `/v3/projects/b75b8a69938b47059767ba2edccff194/users/50484199e9e54240b01e7ecda02e9fa2/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:34.361394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9819b4-de71-4319-922b-526c1c05f7a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b75b8a69938b47059767ba2edccff194, user_id=50484199e9e54240b01e7ecda02e9fa2, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:34.366207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c9819b4-de71-4319-922b-526c1c05f7a5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:34.406743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 815/1638] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:34 2022] PUT /identity/v3/projects/b75b8a69938b47059767ba2edccff194/users/50484199e9e54240b01e7ecda02e9fa2/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:34.409162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9819b4-de71-4319-922b-526c1c05f7a5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:34.427542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcdd8fe-37b9-4994-bde6-1213d8252308 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:34.443160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:34.525140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c9819b4-de71-4319-922b-526c1c05f7a5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:34.532593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd544ba-e11c-4d93-8cec-87a440f96a1d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:34.534907 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd544ba-e11c-4d93-8cec-87a440f96a1d service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:34.542339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd544ba-e11c-4d93-8cec-87a440f96a1d service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:34.542750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd544ba-e11c-4d93-8cec-87a440f96a1d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:34.548829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8561f77-a83e-4ccf-b7f6-4dee54d07b97 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:34.549618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8561f77-a83e-4ccf-b7f6-4dee54d07b97 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:34.549957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8561f77-a83e-4ccf-b7f6-4dee54d07b97 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:34.550252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8561f77-a83e-4ccf-b7f6-4dee54d07b97 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:34.701643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8561f77-a83e-4ccf-b7f6-4dee54d07b97 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:34.702812 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8561f77-a83e-4ccf-b7f6-4dee54d07b97 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:34.716300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd544ba-e11c-4d93-8cec-87a440f96a1d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:34.718933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd544ba-e11c-4d93-8cec-87a440f96a1d service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:34.720758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 824/1639] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:34.722262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73b5ed-0db8-451a-b09f-2c65379bd046 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:34.722551 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73b5ed-0db8-451a-b09f-2c65379bd046 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:34.722830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e73b5ed-0db8-451a-b09f-2c65379bd046 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:34.759063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 816/1640] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:34.770370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9e73b5ed-0db8-451a-b09f-2c65379bd046 None None] MFA Rules not processed for user `50484199e9e54240b01e7ecda02e9fa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:34.771929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9e73b5ed-0db8-451a-b09f-2c65379bd046 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:34.811266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd544ba-e11c-4d93-8cec-87a440f96a1d service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:34.824249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd544ba-e11c-4d93-8cec-87a440f96a1d service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:34.837883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66844b-7c2a-49ab-834b-d702921c3267 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:34.840724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66844b-7c2a-49ab-834b-d702921c3267 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:34.840963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66844b-7c2a-49ab-834b-d702921c3267 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:34.842439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f66844b-7c2a-49ab-834b-d702921c3267 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:34.861379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66844b-7c2a-49ab-834b-d702921c3267 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:34.872674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f66844b-7c2a-49ab-834b-d702921c3267 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:34.893211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:34.923399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 817/1641] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:23:34.932797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 825/1642] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:34 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:35.798877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66844b-7c2a-49ab-834b-d702921c3267 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:35.823717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f66844b-7c2a-49ab-834b-d702921c3267 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:35.832038 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403182de-16df-4d04-8c61-0b833f165c86 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:35.835301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403182de-16df-4d04-8c61-0b833f165c86 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:35.835301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403182de-16df-4d04-8c61-0b833f165c86 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:35.835566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403182de-16df-4d04-8c61-0b833f165c86 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:35.883490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403182de-16df-4d04-8c61-0b833f165c86 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:35.886155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403182de-16df-4d04-8c61-0b833f165c86 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:35.931247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 818/1643] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:35 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:36.065756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73b5ed-0db8-451a-b09f-2c65379bd046 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:36.085099 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e73b5ed-0db8-451a-b09f-2c65379bd046 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:36.105098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62c6c8e-b25f-4d81-8bb4-075670a3a45a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:36.105801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c6c8e-b25f-4d81-8bb4-075670a3a45a service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:36.106089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c6c8e-b25f-4d81-8bb4-075670a3a45a service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:36.109379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62c6c8e-b25f-4d81-8bb4-075670a3a45a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:36.128143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403182de-16df-4d04-8c61-0b833f165c86 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:36.148926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403182de-16df-4d04-8c61-0b833f165c86 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:36.168261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adac47-a4a9-4e23-945c-f096068ce0da service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:36.169119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adac47-a4a9-4e23-945c-f096068ce0da service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:36.169409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adac47-a4a9-4e23-945c-f096068ce0da service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:36.169716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73adac47-a4a9-4e23-945c-f096068ce0da service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:36.284739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:36.324733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c6c8e-b25f-4d81-8bb4-075670a3a45a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:36.327425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62c6c8e-b25f-4d81-8bb4-075670a3a45a service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:36.328802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adac47-a4a9-4e23-945c-f096068ce0da service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:36.330310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73adac47-a4a9-4e23-945c-f096068ce0da service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:36.367299 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 819/1644] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:36 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 239 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:36.373359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 826/1645] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:36 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:40.058065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adac47-a4a9-4e23-945c-f096068ce0da service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:40.303662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73adac47-a4a9-4e23-945c-f096068ce0da service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:40.319254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:40.324055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:40.324055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:40.324293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] PATH_INFO: `/v3/users/d3f8f0bd2cbb4ae9b51990b6efec2b92` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:40.329043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d3f8f0bd2cbb4ae9b51990b6efec2b92)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:40.335455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:40.439216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd3f8f0bd2cbb4ae9b51990b6efec2b92'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:40.458534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd3f8f0bd2cbb4ae9b51990b6efec2b92'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:40.485654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd3f8f0bd2cbb4ae9b51990b6efec2b92'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:40.486415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd3f8f0bd2cbb4ae9b51990b6efec2b92'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:40.487227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd3f8f0bd2cbb4ae9b51990b6efec2b92'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:40.512361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 820/1646] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:40 2022] DELETE /identity/v3/users/d3f8f0bd2cbb4ae9b51990b6efec2b92 => generated 0 bytes in 455 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:40.517456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e53345-70ad-4386-833c-cac7bc6a454f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:40.517939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e53345-70ad-4386-833c-cac7bc6a454f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:40.518042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e53345-70ad-4386-833c-cac7bc6a454f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:40.584328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a4e53345-70ad-4386-833c-cac7bc6a454f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:40.585665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a4e53345-70ad-4386-833c-cac7bc6a454f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:40.862412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 827/1647] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:41.032706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:41.035362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:41.065676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:41.250494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e063e11-d969-4fc4-87d6-e4ad891a5729 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:41.266064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8ec7e2-1264-45d5-8632-0c10e0a00876 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:41.267004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec7e2-1264-45d5-8632-0c10e0a00876 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.267268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec7e2-1264-45d5-8632-0c10e0a00876 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.267545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8ec7e2-1264-45d5-8632-0c10e0a00876 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.279340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ec7e2-1264-45d5-8632-0c10e0a00876 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:41.280826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8ec7e2-1264-45d5-8632-0c10e0a00876 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:41.293536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 821/1648] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 261 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:41.797076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6540fa-d71b-444b-8d00-a47580edc158 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:41.797980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6540fa-d71b-444b-8d00-a47580edc158 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:41.798407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6540fa-d71b-444b-8d00-a47580edc158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:41.852463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8c6540fa-d71b-444b-8d00-a47580edc158 None None] MFA Rules not processed for user `beb901faa8e9410484f83e47e5bcfa8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:41.853687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8c6540fa-d71b-444b-8d00-a47580edc158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:42.083644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 828/1649] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:41 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:42.096467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88e0c7-5623-43aa-a173-ef2067c0de8c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.097205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88e0c7-5623-43aa-a173-ef2067c0de8c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.097452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea88e0c7-5623-43aa-a173-ef2067c0de8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.132775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ea88e0c7-5623-43aa-a173-ef2067c0de8c None None] MFA Rules not processed for user `beb901faa8e9410484f83e47e5bcfa8c`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:42.138623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ea88e0c7-5623-43aa-a173-ef2067c0de8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:42.183502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 822/1650] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:42.195529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6540fa-d71b-444b-8d00-a47580edc158 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.199836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8c6540fa-d71b-444b-8d00-a47580edc158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:42.315295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88e0c7-5623-43aa-a173-ef2067c0de8c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.316841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ea88e0c7-5623-43aa-a173-ef2067c0de8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:42.333970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea88e0c7-5623-43aa-a173-ef2067c0de8c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.340858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.342101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.343985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.344376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] PATH_INFO: `/v3/users/3c1e5bfe31b348d1891206de7e4706eb` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.346670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c1e5bfe31b348d1891206de7e4706eb)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.349546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.397372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6540fa-d71b-444b-8d00-a47580edc158 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.407557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e50242-df4a-4ca4-9eed-a44f7f468d79 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.408651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e50242-df4a-4ca4-9eed-a44f7f468d79 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.413576 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e50242-df4a-4ca4-9eed-a44f7f468d79 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.414201 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e50242-df4a-4ca4-9eed-a44f7f468d79 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.440113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e50242-df4a-4ca4-9eed-a44f7f468d79 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.443237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24e50242-df4a-4ca4-9eed-a44f7f468d79 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.447827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c1e5bfe31b348d1891206de7e4706eb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:42.482267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 829/1651] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:42.492291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c1e5bfe31b348d1891206de7e4706eb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:42.507564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c1e5bfe31b348d1891206de7e4706eb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:42.508816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c1e5bfe31b348d1891206de7e4706eb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:42.509692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c1e5bfe31b348d1891206de7e4706eb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:42.536150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3118f-f818-41f7-a6b6-cd19d16681dc None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.536757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3118f-f818-41f7-a6b6-cd19d16681dc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.537146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd3118f-f818-41f7-a6b6-cd19d16681dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.538739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 823/1652] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:42 2022] DELETE /identity/v3/users/3c1e5bfe31b348d1891206de7e4706eb => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:42.545603 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:42.565868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8cd3118f-f818-41f7-a6b6-cd19d16681dc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:42.567359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8cd3118f-f818-41f7-a6b6-cd19d16681dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:42.700217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 830/1653] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:42.716211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.720578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:42.781071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3118f-f818-41f7-a6b6-cd19d16681dc None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:42.782497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8cd3118f-f818-41f7-a6b6-cd19d16681dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:42.801174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd3118f-f818-41f7-a6b6-cd19d16681dc None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.819618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.821058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.821459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.821726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] PATH_INFO: `/v3/projects/7fedcb34f863465890f3b64591063546` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.823630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7fedcb34f863465890f3b64591063546)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.825611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.870543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f613615d-a779-42a1-95c4-c64e747ff70e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:42.879552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a3acde-434d-4592-8299-9276e631c112 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:42.880233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3acde-434d-4592-8299-9276e631c112 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.880516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3acde-434d-4592-8299-9276e631c112 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.880795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a3acde-434d-4592-8299-9276e631c112 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:42.902031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3acde-434d-4592-8299-9276e631c112 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:42.903757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a3acde-434d-4592-8299-9276e631c112 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:42.916279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 824/1654] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:42.931206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7fedcb34f863465890f3b64591063546', 'request_id': 'req-28722679-bf3d-4ede-acdd-60b7520860f3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:42.940533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7fedcb34f863465890f3b64591063546', 'request_id': 'req-28722679-bf3d-4ede-acdd-60b7520860f3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:42.950328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7fedcb34f863465890f3b64591063546', 'request_id': 'req-28722679-bf3d-4ede-acdd-60b7520860f3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:42.971482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] The token cache is being invalidate because project 7fedcb34f863465890f3b64591063546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:23:42.973122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:42.973978 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-28722679-bf3d-4ede-acdd-60b7520860f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:42.977375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 831/1655] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:23:42 2022] DELETE /identity/v3/projects/7fedcb34f863465890f3b64591063546 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:23:42.986323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ef131-faa2-423d-af63-dcc50a11d603 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:42.986728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ef131-faa2-423d-af63-dcc50a11d603 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:42.987012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb9ef131-faa2-423d-af63-dcc50a11d603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.027410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cb9ef131-faa2-423d-af63-dcc50a11d603 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:43.028670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cb9ef131-faa2-423d-af63-dcc50a11d603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:43.053328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:43.140913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 825/1656] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:43.142627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46c73b-eedc-43ad-bf2d-c8a6b71e045f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.142627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46c73b-eedc-43ad-bf2d-c8a6b71e045f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.142799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a46c73b-eedc-43ad-bf2d-c8a6b71e045f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.182867 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-2a46c73b-eedc-43ad-bf2d-c8a6b71e045f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:43.184028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2a46c73b-eedc-43ad-bf2d-c8a6b71e045f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:43.203607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 832/1657] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 63 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:43.209443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ef131-faa2-423d-af63-dcc50a11d603 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.232255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb9ef131-faa2-423d-af63-dcc50a11d603 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.251415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14673b4e-3ba2-4338-8776-e13694b458b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.255106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14673b4e-3ba2-4338-8776-e13694b458b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.255407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14673b4e-3ba2-4338-8776-e13694b458b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.255688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14673b4e-3ba2-4338-8776-e13694b458b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.260461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14673b4e-3ba2-4338-8776-e13694b458b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:43.260461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14673b4e-3ba2-4338-8776-e13694b458b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.262989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14673b4e-3ba2-4338-8776-e13694b458b1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.308592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 826/1658] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:43.312925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46c73b-eedc-43ad-bf2d-c8a6b71e045f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.326010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a46c73b-eedc-43ad-bf2d-c8a6b71e045f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.345207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc0f9e0-0c74-4783-a039-1c70b6e38ce3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.345936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0f9e0-0c74-4783-a039-1c70b6e38ce3 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.346222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0f9e0-0c74-4783-a039-1c70b6e38ce3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.346502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc0f9e0-0c74-4783-a039-1c70b6e38ce3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.347227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0f9e0-0c74-4783-a039-1c70b6e38ce3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.349375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc0f9e0-0c74-4783-a039-1c70b6e38ce3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.411869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 833/1659] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:43 2022] POST /identity/v3/projects => generated 403 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:43.416664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14673b4e-3ba2-4338-8776-e13694b458b1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.438069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14673b4e-3ba2-4338-8776-e13694b458b1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.572165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673c88fc-2a00-4056-836f-662f10597a16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.572946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c88fc-2a00-4056-836f-662f10597a16 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.573241 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c88fc-2a00-4056-836f-662f10597a16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.573672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673c88fc-2a00-4056-836f-662f10597a16 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.574279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c88fc-2a00-4056-836f-662f10597a16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.576583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673c88fc-2a00-4056-836f-662f10597a16 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.627588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538db0b4-c9e8-44b2-9737-527671d55728 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.628036 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538db0b4-c9e8-44b2-9737-527671d55728 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.628252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538db0b4-c9e8-44b2-9737-527671d55728 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.629564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 834/1660] 10.222.0.69 () {54 vars in 912 bytes} [Wed Jul 20 19:23:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 20 19:23:43.635229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 827/1661] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:43 2022] POST /identity/v3/users => generated 302 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:43.639530 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538db0b4-c9e8-44b2-9737-527671d55728 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.641417 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef298999-e36b-4e34-b5ff-554db0afc1d4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.641745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef298999-e36b-4e34-b5ff-554db0afc1d4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.642028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef298999-e36b-4e34-b5ff-554db0afc1d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.693742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538db0b4-c9e8-44b2-9737-527671d55728 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.710328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a84b26-193b-4dee-aefa-c38a6ff467ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.711552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a84b26-193b-4dee-aefa-c38a6ff467ff admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.711924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a84b26-193b-4dee-aefa-c38a6ff467ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.712193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a84b26-193b-4dee-aefa-c38a6ff467ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.713103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a84b26-193b-4dee-aefa-c38a6ff467ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.714795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a84b26-193b-4dee-aefa-c38a6ff467ff admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.753470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 835/1662] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.754463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a84b26-193b-4dee-aefa-c38a6ff467ff admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.776233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ef298999-e36b-4e34-b5ff-554db0afc1d4 None None] MFA Rules not processed for user `3a1a88089d384964bd9774eda4c5d8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:43.777480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ef298999-e36b-4e34-b5ff-554db0afc1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:43.788538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a84b26-193b-4dee-aefa-c38a6ff467ff admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.807277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e9ad5-0eb7-4491-ab28-da111c8087d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.808095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e9ad5-0eb7-4491-ab28-da111c8087d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.808532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e9ad5-0eb7-4491-ab28-da111c8087d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.808954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129e9ad5-0eb7-4491-ab28-da111c8087d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.810142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e9ad5-0eb7-4491-ab28-da111c8087d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.813436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129e9ad5-0eb7-4491-ab28-da111c8087d8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.836520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 836/1663] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:43.840907 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e9ad5-0eb7-4491-ab28-da111c8087d8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.861518 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129e9ad5-0eb7-4491-ab28-da111c8087d8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:43.873538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34415871-f388-4e04-a5aa-d36afeaef326 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:43.876852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34415871-f388-4e04-a5aa-d36afeaef326 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.877882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34415871-f388-4e04-a5aa-d36afeaef326 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.878826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34415871-f388-4e04-a5aa-d36afeaef326 admin admin] PATH_INFO: `/v3/projects/c30ceaf66d2b484c89dd487b72a62552/users/3f68c2637d4e435b979d68636ecf5789/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.912465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34415871-f388-4e04-a5aa-d36afeaef326 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c30ceaf66d2b484c89dd487b72a62552, user_id=3f68c2637d4e435b979d68636ecf5789, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:43.921490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34415871-f388-4e04-a5aa-d36afeaef326 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:43.922810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 828/1664] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 19:23:43 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:23:43.946121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 837/1665] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:43 2022] PUT /identity/v3/projects/c30ceaf66d2b484c89dd487b72a62552/users/3f68c2637d4e435b979d68636ecf5789/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:23:43.947825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef298999-e36b-4e34-b5ff-554db0afc1d4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:43.956233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ef298999-e36b-4e34-b5ff-554db0afc1d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:43.957320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de7ff4-7c22-4184-b3c9-ea94a1c8df46 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:43.957786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de7ff4-7c22-4184-b3c9-ea94a1c8df46 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:43.958213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43de7ff4-7c22-4184-b3c9-ea94a1c8df46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:43.971147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef298999-e36b-4e34-b5ff-554db0afc1d4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:44.042193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:44.050302 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:44.072214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-43de7ff4-7c22-4184-b3c9-ea94a1c8df46 None None] MFA Rules not processed for user `3f68c2637d4e435b979d68636ecf5789`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:44.077428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-43de7ff4-7c22-4184-b3c9-ea94a1c8df46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:44.085680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad56fa9-6a24-4898-8571-83dc0889ecf8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:44.088332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56fa9-6a24-4898-8571-83dc0889ecf8 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.089052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56fa9-6a24-4898-8571-83dc0889ecf8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.089498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad56fa9-6a24-4898-8571-83dc0889ecf8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.107830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad56fa9-6a24-4898-8571-83dc0889ecf8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:44.109308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad56fa9-6a24-4898-8571-83dc0889ecf8 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:44.130410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 829/1666] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:43 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:44.216721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 838/1667] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:43 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:44.222329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad56fa9-6a24-4898-8571-83dc0889ecf8 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:44.348195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad56fa9-6a24-4898-8571-83dc0889ecf8 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:44.357243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02a144-cfcb-4131-b6a8-04d3ccdaa1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:44.358398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02a144-cfcb-4131-b6a8-04d3ccdaa1cb admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.359444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02a144-cfcb-4131-b6a8-04d3ccdaa1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.360106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02a144-cfcb-4131-b6a8-04d3ccdaa1cb admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.361013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02a144-cfcb-4131-b6a8-04d3ccdaa1cb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:44.364108 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f02a144-cfcb-4131-b6a8-04d3ccdaa1cb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:44.432832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 830/1668] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:44 2022] POST /identity/v3/projects => generated 403 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:23:44.434297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de7ff4-7c22-4184-b3c9-ea94a1c8df46 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:44.463876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43de7ff4-7c22-4184-b3c9-ea94a1c8df46 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:44.613294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40618257-0ad3-44a2-9a47-7a6b720f679e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:44.622288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40618257-0ad3-44a2-9a47-7a6b720f679e admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.622582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40618257-0ad3-44a2-9a47-7a6b720f679e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.622862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40618257-0ad3-44a2-9a47-7a6b720f679e admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.626428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40618257-0ad3-44a2-9a47-7a6b720f679e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:44.634252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40618257-0ad3-44a2-9a47-7a6b720f679e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:44.699539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 839/1669] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:44 2022] POST /identity/v3/users => generated 308 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:44.702945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02a144-cfcb-4131-b6a8-04d3ccdaa1cb admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:44.738688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f02a144-cfcb-4131-b6a8-04d3ccdaa1cb admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:44.748194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21c83f-869a-4c56-b0f0-203b512a718d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:44.749319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21c83f-869a-4c56-b0f0-203b512a718d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.749772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21c83f-869a-4c56-b0f0-203b512a718d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.750180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a21c83f-869a-4c56-b0f0-203b512a718d admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.751094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21c83f-869a-4c56-b0f0-203b512a718d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:44.764235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a21c83f-869a-4c56-b0f0-203b512a718d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:44.801625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 831/1670] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:44.805963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40618257-0ad3-44a2-9a47-7a6b720f679e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:44.829166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40618257-0ad3-44a2-9a47-7a6b720f679e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:44.845157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f9842-4d55-4240-a77a-90c6465781da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:44.848384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f9842-4d55-4240-a77a-90c6465781da admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:44.850086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f9842-4d55-4240-a77a-90c6465781da admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:44.850726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376f9842-4d55-4240-a77a-90c6465781da admin admin] PATH_INFO: `/v3/projects/e9f126ae433048698e1504e7f55c6805/users/89b9172fcf27430393b8f1738dc9a1e8/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:44.904454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f9842-4d55-4240-a77a-90c6465781da admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f126ae433048698e1504e7f55c6805, user_id=89b9172fcf27430393b8f1738dc9a1e8, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:44.913541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376f9842-4d55-4240-a77a-90c6465781da admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:44.952915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 840/1671] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:44 2022] PUT /identity/v3/projects/e9f126ae433048698e1504e7f55c6805/users/89b9172fcf27430393b8f1738dc9a1e8/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:23:44.955614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21c83f-869a-4c56-b0f0-203b512a718d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:44.978724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a21c83f-869a-4c56-b0f0-203b512a718d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.063304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f9842-4d55-4240-a77a-90c6465781da admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.066535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-376f9842-4d55-4240-a77a-90c6465781da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:45.168692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461e2c1-420e-48e3-af85-b11ba361982a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.169983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461e2c1-420e-48e3-af85-b11ba361982a admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.170411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461e2c1-420e-48e3-af85-b11ba361982a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.170824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3461e2c1-420e-48e3-af85-b11ba361982a admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.172126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461e2c1-420e-48e3-af85-b11ba361982a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.175354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3461e2c1-420e-48e3-af85-b11ba361982a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.216023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 832/1672] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:44 2022] GET /identity/v3/roles => generated 1420 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:45.221469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461e2c1-420e-48e3-af85-b11ba361982a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.262063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376f9842-4d55-4240-a77a-90c6465781da admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.267818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.268952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.269309 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.269698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] PATH_INFO: `/v3/projects/53495064e7074f4a9d5fc05907272405` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.274400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53495064e7074f4a9d5fc05907272405)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.279603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3461e2c1-420e-48e3-af85-b11ba361982a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.287666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b089665a-6f62-4f24-8bd3-24fd00c3d7e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.288836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.289911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089665a-6f62-4f24-8bd3-24fd00c3d7e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.290451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089665a-6f62-4f24-8bd3-24fd00c3d7e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.290875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b089665a-6f62-4f24-8bd3-24fd00c3d7e6 admin admin] PATH_INFO: `/v3/projects/e9f126ae433048698e1504e7f55c6805/users/89b9172fcf27430393b8f1738dc9a1e8/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.293694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089665a-6f62-4f24-8bd3-24fd00c3d7e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f126ae433048698e1504e7f55c6805, user_id=89b9172fcf27430393b8f1738dc9a1e8, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.297748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b089665a-6f62-4f24-8bd3-24fd00c3d7e6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:45.311632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:45.317419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 833/1673] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:45 2022] PUT /identity/v3/projects/e9f126ae433048698e1504e7f55c6805/users/89b9172fcf27430393b8f1738dc9a1e8/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:45.322703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e17d2-761c-40c4-b832-65b7bb298c2a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.323334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e17d2-761c-40c4-b832-65b7bb298c2a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.323815 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b9e17d2-761c-40c4-b832-65b7bb298c2a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.326867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 834/1674] 10.222.0.69 () {54 vars in 961 bytes} [Wed Jul 20 19:23:45 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:45.330338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a9adb-01a5-4aec-8a38-af734b871e65 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.331030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a9adb-01a5-4aec-8a38-af734b871e65 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.332033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235a9adb-01a5-4aec-8a38-af734b871e65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.411614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-235a9adb-01a5-4aec-8a38-af734b871e65 None None] MFA Rules not processed for user `89b9172fcf27430393b8f1738dc9a1e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:45.419644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-235a9adb-01a5-4aec-8a38-af734b871e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:45.473317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:45.488330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53495064e7074f4a9d5fc05907272405', 'request_id': 'req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:45.499856 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53495064e7074f4a9d5fc05907272405', 'request_id': 'req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:45.510442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53495064e7074f4a9d5fc05907272405', 'request_id': 'req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:45.540434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] The token cache is being invalidate because project 53495064e7074f4a9d5fc05907272405 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:23:45.540896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:45.541818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-508154c7-3338-4ea3-ae2d-5ca3b9b461a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:45.549206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 841/1675] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:23:45 2022] DELETE /identity/v3/projects/53495064e7074f4a9d5fc05907272405 => generated 0 bytes in 483 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:45.549206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c83c2-9b2e-413b-a11a-ab7ba5ec221d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.549206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c83c2-9b2e-413b-a11a-ab7ba5ec221d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.549206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84c83c2-9b2e-413b-a11a-ab7ba5ec221d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.620917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f84c83c2-9b2e-413b-a11a-ab7ba5ec221d None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:45.622908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f84c83c2-9b2e-413b-a11a-ab7ba5ec221d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:45.624340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 835/1676] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:45 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 295 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:45.625234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a9adb-01a5-4aec-8a38-af734b871e65 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.628983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-235a9adb-01a5-4aec-8a38-af734b871e65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:45.845870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 842/1677] 10.222.0.69 () {58 vars in 1058 bytes} [Wed Jul 20 19:23:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:45.845870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c83c2-9b2e-413b-a11a-ab7ba5ec221d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:45.850181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f84c83c2-9b2e-413b-a11a-ab7ba5ec221d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:45.862099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235a9adb-01a5-4aec-8a38-af734b871e65 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:45.869290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:45.870722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:45.871853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:45.872332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] PATH_INFO: `/v3/users/50484199e9e54240b01e7ecda02e9fa2` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:45.875665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50484199e9e54240b01e7ecda02e9fa2)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:45.879236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:46.018937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50484199e9e54240b01e7ecda02e9fa2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:46.032402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84c83c2-9b2e-413b-a11a-ab7ba5ec221d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:46.041991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1316af-94d9-4b66-a6ff-7c284dfb5d0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:46.042668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1316af-94d9-4b66-a6ff-7c284dfb5d0e service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.042986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1316af-94d9-4b66-a6ff-7c284dfb5d0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.045143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c1316af-94d9-4b66-a6ff-7c284dfb5d0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.053470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50484199e9e54240b01e7ecda02e9fa2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:46.066775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50484199e9e54240b01e7ecda02e9fa2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:46.067736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50484199e9e54240b01e7ecda02e9fa2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:46.068708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-cf583b0f-cba5-4791-9557-a919b1b89055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50484199e9e54240b01e7ecda02e9fa2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:46.070430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1316af-94d9-4b66-a6ff-7c284dfb5d0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:46.075492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c1316af-94d9-4b66-a6ff-7c284dfb5d0e service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:46.105260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 843/1678] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:45 2022] GET /identity/v3/auth/tokens => generated 3337 bytes in 260 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:46.105675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1316af-94d9-4b66-a6ff-7c284dfb5d0e service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:46.123150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 836/1679] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:45 2022] DELETE /identity/v3/users/50484199e9e54240b01e7ecda02e9fa2 => generated 0 bytes in 495 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:46.147633 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c1316af-94d9-4b66-a6ff-7c284dfb5d0e service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:46.272241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:46.383504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecef73b-86b6-438d-83f7-2e4a11899048 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:46.385553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef73b-86b6-438d-83f7-2e4a11899048 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.385952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef73b-86b6-438d-83f7-2e4a11899048 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.387084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecef73b-86b6-438d-83f7-2e4a11899048 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.391764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7ecef73b-86b6-438d-83f7-2e4a11899048 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:46.415526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecef73b-86b6-438d-83f7-2e4a11899048 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:46.417263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecef73b-86b6-438d-83f7-2e4a11899048 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:46.434838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec3c2d-98e3-4d7f-b66f-c69867f05700 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.435760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec3c2d-98e3-4d7f-b66f-c69867f05700 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.436247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ec3c2d-98e3-4d7f-b66f-c69867f05700 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.462163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 844/1680] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:46.494691 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b7ec3c2d-98e3-4d7f-b66f-c69867f05700 None None] MFA Rules not processed for user `343e480dd5ee4389ba430129cd7daaa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:46.495942 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b7ec3c2d-98e3-4d7f-b66f-c69867f05700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:46.546507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:46.663070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 837/1681] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:46 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:46.680933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecef73b-86b6-438d-83f7-2e4a11899048 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:46.682553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7ecef73b-86b6-438d-83f7-2e4a11899048 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:46.819557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecef73b-86b6-438d-83f7-2e4a11899048 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:46.838269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c5a5f-5d66-4ceb-8422-3ff507da2e17 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:46.839078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c5a5f-5d66-4ceb-8422-3ff507da2e17 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:46.839468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c5a5f-5d66-4ceb-8422-3ff507da2e17 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:46.839847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4c5a5f-5d66-4ceb-8422-3ff507da2e17 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:46.856119 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c5a5f-5d66-4ceb-8422-3ff507da2e17 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:46.858067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4c5a5f-5d66-4ceb-8422-3ff507da2e17 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:46.871144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 845/1682] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:46 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:48.333174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec3c2d-98e3-4d7f-b66f-c69867f05700 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:48.335366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b7ec3c2d-98e3-4d7f-b66f-c69867f05700 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:48.392454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c5a5f-5d66-4ceb-8422-3ff507da2e17 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:48.394170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fd4c5a5f-5d66-4ceb-8422-3ff507da2e17 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:48.539984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4c5a5f-5d66-4ceb-8422-3ff507da2e17 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:48.545377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:48.546529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:48.547109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:48.547534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] PATH_INFO: `/v3/projects/b75b8a69938b47059767ba2edccff194` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:48.548999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b75b8a69938b47059767ba2edccff194)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:48.551295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:48.573724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ec3c2d-98e3-4d7f-b66f-c69867f05700 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:48.591149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:48.592070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:48.593205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:48.595027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] PATH_INFO: `/v3/users/3f68c2637d4e435b979d68636ecf5789` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:48.600277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f68c2637d4e435b979d68636ecf5789)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:48.610957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:48.633804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b75b8a69938b47059767ba2edccff194', 'request_id': 'req-a5138233-3297-4d07-a7db-2cc7f6491405'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:48.645367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:48.664953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b75b8a69938b47059767ba2edccff194', 'request_id': 'req-a5138233-3297-4d07-a7db-2cc7f6491405'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:48.685622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b75b8a69938b47059767ba2edccff194', 'request_id': 'req-a5138233-3297-4d07-a7db-2cc7f6491405'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:48.714588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] The token cache is being invalidate because project b75b8a69938b47059767ba2edccff194 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:23:48.715345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:48.719374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a5138233-3297-4d07-a7db-2cc7f6491405 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:48.731699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 846/1683] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:23:48 2022] DELETE /identity/v3/projects/b75b8a69938b47059767ba2edccff194 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:48.745115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae25627-2d42-463f-9027-85b24cf60c39 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:48.745794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae25627-2d42-463f-9027-85b24cf60c39 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:48.746368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae25627-2d42-463f-9027-85b24cf60c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:48.761645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f68c2637d4e435b979d68636ecf5789'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:48.776728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-eae25627-2d42-463f-9027-85b24cf60c39 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:48.778304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-eae25627-2d42-463f-9027-85b24cf60c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:48.796841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f68c2637d4e435b979d68636ecf5789'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:48.816234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f68c2637d4e435b979d68636ecf5789'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:48.817127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f68c2637d4e435b979d68636ecf5789'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:48.817843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a0e2471d-4471-454c-b824-f8b4830d6fc1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f68c2637d4e435b979d68636ecf5789'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:48.849172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 838/1684] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:48 2022] DELETE /identity/v3/users/3f68c2637d4e435b979d68636ecf5789 => generated 0 bytes in 516 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:48.855328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393df71-62bf-4442-908a-8fa2dcc9450c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:48.855810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393df71-62bf-4442-908a-8fa2dcc9450c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:48.856159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1393df71-62bf-4442-908a-8fa2dcc9450c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:48.895561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1393df71-62bf-4442-908a-8fa2dcc9450c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:48.897962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1393df71-62bf-4442-908a-8fa2dcc9450c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:48.926654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 847/1685] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:48.943736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17a39-434e-45ee-a2ad-117c688cb60d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:48.944394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17a39-434e-45ee-a2ad-117c688cb60d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:48.944947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf17a39-434e-45ee-a2ad-117c688cb60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:48.989744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-dbf17a39-434e-45ee-a2ad-117c688cb60d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:48.991387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dbf17a39-434e-45ee-a2ad-117c688cb60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:49.020551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:49.078286 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 839/1686] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:49.108829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 848/1687] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:49.113614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393df71-62bf-4442-908a-8fa2dcc9450c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.128657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1393df71-62bf-4442-908a-8fa2dcc9450c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.136866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a2ae8-728e-40d0-9880-d1f2ffb5acc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.137891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a2ae8-728e-40d0-9880-d1f2ffb5acc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.138249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a2ae8-728e-40d0-9880-d1f2ffb5acc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.138599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333a2ae8-728e-40d0-9880-d1f2ffb5acc0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.139475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a2ae8-728e-40d0-9880-d1f2ffb5acc0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:49.140028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a2ae8-728e-40d0-9880-d1f2ffb5acc0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.142043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333a2ae8-728e-40d0-9880-d1f2ffb5acc0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.145990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf17a39-434e-45ee-a2ad-117c688cb60d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.148525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dbf17a39-434e-45ee-a2ad-117c688cb60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:49.207063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 840/1688] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:23:49.211802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a2ae8-728e-40d0-9880-d1f2ffb5acc0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.239162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333a2ae8-728e-40d0-9880-d1f2ffb5acc0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.253972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1c53b-aab6-4369-aed6-e971893025b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.256710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1c53b-aab6-4369-aed6-e971893025b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.257090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1c53b-aab6-4369-aed6-e971893025b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.258317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e1c53b-aab6-4369-aed6-e971893025b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.259132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1c53b-aab6-4369-aed6-e971893025b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.277560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e1c53b-aab6-4369-aed6-e971893025b4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.355308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf17a39-434e-45ee-a2ad-117c688cb60d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.359084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 841/1689] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:49 2022] POST /identity/v3/projects => generated 381 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:49.362580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1c53b-aab6-4369-aed6-e971893025b4 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.482485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:49.545283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee393a-a985-41f3-8de4-b32f6841ebd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.545925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee393a-a985-41f3-8de4-b32f6841ebd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.546216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee393a-a985-41f3-8de4-b32f6841ebd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.546493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ee393a-a985-41f3-8de4-b32f6841ebd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.578761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e1c53b-aab6-4369-aed6-e971893025b4 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.596943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fdaed-b525-49de-b6bc-6509b0e1571c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.597690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fdaed-b525-49de-b6bc-6509b0e1571c admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.597975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fdaed-b525-49de-b6bc-6509b0e1571c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.598383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7fdaed-b525-49de-b6bc-6509b0e1571c admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.599022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fdaed-b525-49de-b6bc-6509b0e1571c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.611971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7fdaed-b525-49de-b6bc-6509b0e1571c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.698078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 842/1690] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:49 2022] POST /identity/v3/users => generated 291 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:49.701176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fdaed-b525-49de-b6bc-6509b0e1571c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.721809 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7fdaed-b525-49de-b6bc-6509b0e1571c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.736054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad9084-7847-4f1d-9c02-852dbacc1226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.736911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad9084-7847-4f1d-9c02-852dbacc1226 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.737204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad9084-7847-4f1d-9c02-852dbacc1226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.737494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aad9084-7847-4f1d-9c02-852dbacc1226 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.738623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad9084-7847-4f1d-9c02-852dbacc1226 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.742074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aad9084-7847-4f1d-9c02-852dbacc1226 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.778302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 843/1691] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:49.784315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad9084-7847-4f1d-9c02-852dbacc1226 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.801822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee393a-a985-41f3-8de4-b32f6841ebd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.808501 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ee393a-a985-41f3-8de4-b32f6841ebd7 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.814896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aad9084-7847-4f1d-9c02-852dbacc1226 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.832201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86278541-0aa1-4bc6-9a30-58bfc9e0c1e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.834018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86278541-0aa1-4bc6-9a30-58bfc9e0c1e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.835144 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86278541-0aa1-4bc6-9a30-58bfc9e0c1e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.837924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86278541-0aa1-4bc6-9a30-58bfc9e0c1e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.839264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86278541-0aa1-4bc6-9a30-58bfc9e0c1e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.843701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86278541-0aa1-4bc6-9a30-58bfc9e0c1e7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.846634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 849/1692] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 701 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:23:49.866902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee393a-a985-41f3-8de4-b32f6841ebd7 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:49.876469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 844/1693] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:49 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:49.881762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ee393a-a985-41f3-8de4-b32f6841ebd7 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:49.895391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d0b6160-48c6-4b29-89d0-ac8d93d237b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:49.897224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b6160-48c6-4b29-89d0-ac8d93d237b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.897765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b6160-48c6-4b29-89d0-ac8d93d237b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.899081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d0b6160-48c6-4b29-89d0-ac8d93d237b4 admin admin] PATH_INFO: `/v3/projects/cc30e4dd5d1f4829b68718ef10261ac6/users/62d7396dc47a4c48a4e581872da239b8/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:49.922552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b6160-48c6-4b29-89d0-ac8d93d237b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc30e4dd5d1f4829b68718ef10261ac6, user_id=62d7396dc47a4c48a4e581872da239b8, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:49.934904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d0b6160-48c6-4b29-89d0-ac8d93d237b4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:49.972200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 850/1694] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:49 2022] PUT /identity/v3/projects/cc30e4dd5d1f4829b68718ef10261ac6/users/62d7396dc47a4c48a4e581872da239b8/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:49.974375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e3c3-155a-447f-9e32-f193505c8bb3 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:49.974662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e3c3-155a-447f-9e32-f193505c8bb3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:49.974945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e34e3c3-155a-447f-9e32-f193505c8bb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:50.028634 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7e34e3c3-155a-447f-9e32-f193505c8bb3 None None] MFA Rules not processed for user `62d7396dc47a4c48a4e581872da239b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:50.031562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7e34e3c3-155a-447f-9e32-f193505c8bb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:50.249938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2b57c-07e2-4800-ac8f-2d5851506869 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:50.250455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2b57c-07e2-4800-ac8f-2d5851506869 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:50.252534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c2b57c-07e2-4800-ac8f-2d5851506869 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:50.336023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-55c2b57c-07e2-4800-ac8f-2d5851506869 None None] MFA Rules not processed for user `9192f2842ca044f08e5f95f98f2e1b88`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:50.350288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 845/1695] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:49 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:50.350288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36a98d-8bde-4f81-a042-e360f7854c7a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:50.350536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-55c2b57c-07e2-4800-ac8f-2d5851506869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:50.350628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36a98d-8bde-4f81-a042-e360f7854c7a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:50.350628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe36a98d-8bde-4f81-a042-e360f7854c7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:50.365192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:50.400465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fe36a98d-8bde-4f81-a042-e360f7854c7a None None] MFA Rules not processed for user `4d08da0ef69e4cf2abc6c18bbbc51a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:50.402105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fe36a98d-8bde-4f81-a042-e360f7854c7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:50.583028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 846/1696] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:50.586348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36a98d-8bde-4f81-a042-e360f7854c7a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:50.655246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 851/1697] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 408 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:50.659442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2b57c-07e2-4800-ac8f-2d5851506869 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:50.662286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-55c2b57c-07e2-4800-ac8f-2d5851506869 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:50.784978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe36a98d-8bde-4f81-a042-e360f7854c7a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:50.820735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eadde0-9637-4fe2-a6b6-e235b2def480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:50.835240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eadde0-9637-4fe2-a6b6-e235b2def480 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:50.835531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eadde0-9637-4fe2-a6b6-e235b2def480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:50.835851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eadde0-9637-4fe2-a6b6-e235b2def480 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:50.836684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eadde0-9637-4fe2-a6b6-e235b2def480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:50.838612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eadde0-9637-4fe2-a6b6-e235b2def480 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:50.929267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 847/1698] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:50 2022] POST /identity/v3/projects => generated 383 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:50.933966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4fc1-5d94-475b-9e49-a800ba1584ef None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:50.934282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4fc1-5d94-475b-9e49-a800ba1584ef None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:50.934527 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2d4fc1-5d94-475b-9e49-a800ba1584ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:50.939963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:50.984912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cc2d4fc1-5d94-475b-9e49-a800ba1584ef None None] MFA Rules not processed for user `9192f2842ca044f08e5f95f98f2e1b88`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:50.990331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cc2d4fc1-5d94-475b-9e49-a800ba1584ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:50.996373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c2b57c-07e2-4800-ac8f-2d5851506869 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:51.165328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 848/1699] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:50 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 234 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:51.168642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de7a13-beb7-4384-8a9c-f8c3020276ac None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:51.168940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de7a13-beb7-4384-8a9c-f8c3020276ac None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:51.169251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58de7a13-beb7-4384-8a9c-f8c3020276ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:51.199792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a21e3-cb02-4a23-a18e-7ba7891166f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:51.202083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a21e3-cb02-4a23-a18e-7ba7891166f0 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:51.205789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a21e3-cb02-4a23-a18e-7ba7891166f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:51.206263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197a21e3-cb02-4a23-a18e-7ba7891166f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:51.237420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-58de7a13-beb7-4384-8a9c-f8c3020276ac None None] MFA Rules not processed for user `343e480dd5ee4389ba430129cd7daaa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:51.238803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-58de7a13-beb7-4384-8a9c-f8c3020276ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:51.313104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:51.399798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a21e3-cb02-4a23-a18e-7ba7891166f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:51.418187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197a21e3-cb02-4a23-a18e-7ba7891166f0 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:51.453756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 849/1700] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:51 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 283 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:51.453756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de7a13-beb7-4384-8a9c-f8c3020276ac None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:51.466456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 852/1701] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 810 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:51.468094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a21e3-cb02-4a23-a18e-7ba7891166f0 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:51.478002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-197a21e3-cb02-4a23-a18e-7ba7891166f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:51.642402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197a21e3-cb02-4a23-a18e-7ba7891166f0 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:51.658485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:51.659413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:51.659697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:51.659973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] PATH_INFO: `/v3/projects/c30ceaf66d2b484c89dd487b72a62552` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:51.669005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c30ceaf66d2b484c89dd487b72a62552)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:51.678562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:51.686061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58de7a13-beb7-4384-8a9c-f8c3020276ac None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:51.704355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380445a9-1d9b-4458-a6b5-8391fc19b1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:51.711397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380445a9-1d9b-4458-a6b5-8391fc19b1e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:51.711928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380445a9-1d9b-4458-a6b5-8391fc19b1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:51.712346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380445a9-1d9b-4458-a6b5-8391fc19b1e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:51.713087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380445a9-1d9b-4458-a6b5-8391fc19b1e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:51.715146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380445a9-1d9b-4458-a6b5-8391fc19b1e4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:51.780321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:51.829340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 850/1702] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:51 2022] POST /identity/v3/users => generated 298 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:51.831855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380445a9-1d9b-4458-a6b5-8391fc19b1e4 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:51.839729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-380445a9-1d9b-4458-a6b5-8391fc19b1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:51.850157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c30ceaf66d2b484c89dd487b72a62552', 'request_id': 'req-45b346a2-3d51-4743-b3cf-efeeaddf89d8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:51.895821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c30ceaf66d2b484c89dd487b72a62552', 'request_id': 'req-45b346a2-3d51-4743-b3cf-efeeaddf89d8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:51.930850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380445a9-1d9b-4458-a6b5-8391fc19b1e4 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:51.933498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c30ceaf66d2b484c89dd487b72a62552', 'request_id': 'req-45b346a2-3d51-4743-b3cf-efeeaddf89d8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:51.983279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] The token cache is being invalidate because project c30ceaf66d2b484c89dd487b72a62552 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:23:51.983626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:51.992390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.005790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 853/1703] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:23:51 2022] DELETE /identity/v3/projects/c30ceaf66d2b484c89dd487b72a62552 => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:52.006774 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:52.014122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.165044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c67b60-e0b4-4b1b-9b76-4086671500ff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.166064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c67b60-e0b4-4b1b-9b76-4086671500ff service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.166698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c67b60-e0b4-4b1b-9b76-4086671500ff service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.167272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c67b60-e0b4-4b1b-9b76-4086671500ff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.169404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c3c67b60-e0b4-4b1b-9b76-4086671500ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.255223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b346a2-3d51-4743-b3cf-efeeaddf89d8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:52.263783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a15bcda-0317-4378-82d2-7b5e28bc61af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.264587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15bcda-0317-4378-82d2-7b5e28bc61af admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.264941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15bcda-0317-4378-82d2-7b5e28bc61af admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.265429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a15bcda-0317-4378-82d2-7b5e28bc61af admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.267396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15bcda-0317-4378-82d2-7b5e28bc61af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:52.272531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a15bcda-0317-4378-82d2-7b5e28bc61af admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:52.309221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 854/1704] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:52.311083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628adcc-e3fc-4790-bc5e-8bf96dd11fb0 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.311378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628adcc-e3fc-4790-bc5e-8bf96dd11fb0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.311655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5628adcc-e3fc-4790-bc5e-8bf96dd11fb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.363371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:52.372860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c67b60-e0b4-4b1b-9b76-4086671500ff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:52.379847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c67b60-e0b4-4b1b-9b76-4086671500ff service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:52.423119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 851/1705] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:51 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 593 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:52.443043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c67b60-e0b4-4b1b-9b76-4086671500ff service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:52.449981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c3c67b60-e0b4-4b1b-9b76-4086671500ff service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.477480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5628adcc-e3fc-4790-bc5e-8bf96dd11fb0 None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:52.483008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5628adcc-e3fc-4790-bc5e-8bf96dd11fb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:52.498749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c67b60-e0b4-4b1b-9b76-4086671500ff service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:52.523127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.524027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.524491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.524879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] PATH_INFO: `/v3/users/89b9172fcf27430393b8f1738dc9a1e8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.530747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=89b9172fcf27430393b8f1738dc9a1e8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:52.532825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:52.679929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '89b9172fcf27430393b8f1738dc9a1e8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.714366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '89b9172fcf27430393b8f1738dc9a1e8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.739711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '89b9172fcf27430393b8f1738dc9a1e8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.743104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '89b9172fcf27430393b8f1738dc9a1e8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.744523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '89b9172fcf27430393b8f1738dc9a1e8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:52.745848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 855/1706] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 19:23:52 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 436 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:52.747717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628adcc-e3fc-4790-bc5e-8bf96dd11fb0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:52.793926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5628adcc-e3fc-4790-bc5e-8bf96dd11fb0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:52.807710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703b6a0-d3f5-42e5-a2b7-e76ba2c409f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:52.808617 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703b6a0-d3f5-42e5-a2b7-e76ba2c409f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:52.808953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703b6a0-d3f5-42e5-a2b7-e76ba2c409f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:52.809315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d703b6a0-d3f5-42e5-a2b7-e76ba2c409f4 admin admin] PATH_INFO: `/v3/projects/ed7a2a31c0da4b0f8022e3f4d29835d8/users/0d26c40913834ebf943708ab23f9cfda/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:52.823796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 852/1707] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:52 2022] DELETE /identity/v3/users/89b9172fcf27430393b8f1738dc9a1e8 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:52.843588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703b6a0-d3f5-42e5-a2b7-e76ba2c409f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7a2a31c0da4b0f8022e3f4d29835d8, user_id=0d26c40913834ebf943708ab23f9cfda, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:52.847568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d703b6a0-d3f5-42e5-a2b7-e76ba2c409f4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:52.875686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 856/1708] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:52 2022] PUT /identity/v3/projects/ed7a2a31c0da4b0f8022e3f4d29835d8/users/0d26c40913834ebf943708ab23f9cfda/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:52.880841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:52.885768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.027082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703b6a0-d3f5-42e5-a2b7-e76ba2c409f4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:53.030024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d703b6a0-d3f5-42e5-a2b7-e76ba2c409f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.036129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:53.095408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906ce001-a4c5-4339-ae71-9218a81ea5c1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:53.270282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d703b6a0-d3f5-42e5-a2b7-e76ba2c409f4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:53.278122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637bd43-a5e3-4b15-a674-1924d134df2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:53.280512 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637bd43-a5e3-4b15-a674-1924d134df2c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:53.281923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637bd43-a5e3-4b15-a674-1924d134df2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:53.284524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e2f6e-3131-45e9-8333-bdb984d9eb3f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:53.285573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e2f6e-3131-45e9-8333-bdb984d9eb3f service swift] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:53.286203 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4637bd43-a5e3-4b15-a674-1924d134df2c admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:53.286494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e2f6e-3131-45e9-8333-bdb984d9eb3f service swift] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:53.287184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84e2f6e-3131-45e9-8333-bdb984d9eb3f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:53.287659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637bd43-a5e3-4b15-a674-1924d134df2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:53.291375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4637bd43-a5e3-4b15-a674-1924d134df2c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:53.292403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b84e2f6e-3131-45e9-8333-bdb984d9eb3f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.319966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 853/1709] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:52 2022] GET /identity/v3/roles => generated 1420 bytes in 440 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:53.327219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637bd43-a5e3-4b15-a674-1924d134df2c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:53.350499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4637bd43-a5e3-4b15-a674-1924d134df2c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:53.363353 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c756b51-ff3c-470f-9e22-267a7d4c97b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:53.364310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c756b51-ff3c-470f-9e22-267a7d4c97b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:53.364670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c756b51-ff3c-470f-9e22-267a7d4c97b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:53.365022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c756b51-ff3c-470f-9e22-267a7d4c97b5 admin admin] PATH_INFO: `/v3/projects/ed7a2a31c0da4b0f8022e3f4d29835d8/users/0d26c40913834ebf943708ab23f9cfda/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:53.371269 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c756b51-ff3c-470f-9e22-267a7d4c97b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed7a2a31c0da4b0f8022e3f4d29835d8, user_id=0d26c40913834ebf943708ab23f9cfda, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:53.375624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c756b51-ff3c-470f-9e22-267a7d4c97b5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:53.389484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:53.404711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 854/1710] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:53 2022] PUT /identity/v3/projects/ed7a2a31c0da4b0f8022e3f4d29835d8/users/0d26c40913834ebf943708ab23f9cfda/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:53.416773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535e56-45ea-417e-81f4-4ffcef636d25 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:53.417075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535e56-45ea-417e-81f4-4ffcef636d25 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:53.418075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e535e56-45ea-417e-81f4-4ffcef636d25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:53.447040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e2f6e-3131-45e9-8333-bdb984d9eb3f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:53.456581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84e2f6e-3131-45e9-8333-bdb984d9eb3f service swift] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:53.500751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 857/1711] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 19:23:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:23:53.514622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3e535e56-45ea-417e-81f4-4ffcef636d25 None None] MFA Rules not processed for user `0d26c40913834ebf943708ab23f9cfda`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:53.553634 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3e535e56-45ea-417e-81f4-4ffcef636d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:53.896411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 855/1712] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:53 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 482 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:53.912040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e2f6e-3131-45e9-8333-bdb984d9eb3f service swift] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:53.914628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b84e2f6e-3131-45e9-8333-bdb984d9eb3f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:54.156111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84e2f6e-3131-45e9-8333-bdb984d9eb3f service swift] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:54.157427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e535e56-45ea-417e-81f4-4ffcef636d25 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:54.164803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3e535e56-45ea-417e-81f4-4ffcef636d25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:54.275374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:54.340618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b93a4-febe-4bef-b80b-dcf7e6330362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:54.341321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b93a4-febe-4bef-b80b-dcf7e6330362 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:54.354484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b93a4-febe-4bef-b80b-dcf7e6330362 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:54.354884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-813b93a4-febe-4bef-b80b-dcf7e6330362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:54.384260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b93a4-febe-4bef-b80b-dcf7e6330362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:54.389023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-813b93a4-febe-4bef-b80b-dcf7e6330362 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:54.432365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 858/1713] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:53 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 521 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:54.529592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e535e56-45ea-417e-81f4-4ffcef636d25 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:54.566030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:54.578533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:54.588381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:54.588381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] PATH_INFO: `/v3/users/28551e3575b24089b0043692f65a8ffd` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:54.589541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=28551e3575b24089b0043692f65a8ffd)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:54.591822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:54.732130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b93a4-febe-4bef-b80b-dcf7e6330362 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:54.749815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-813b93a4-febe-4bef-b80b-dcf7e6330362 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:54.774792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc54e2e2-404b-43da-befe-d07c75e982c1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:54.775590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54e2e2-404b-43da-befe-d07c75e982c1 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:54.775878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54e2e2-404b-43da-befe-d07c75e982c1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:54.776163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc54e2e2-404b-43da-befe-d07c75e982c1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:54.783765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cc54e2e2-404b-43da-befe-d07c75e982c1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:54.823424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '28551e3575b24089b0043692f65a8ffd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:54.898680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '28551e3575b24089b0043692f65a8ffd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:54.946265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '28551e3575b24089b0043692f65a8ffd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:54.947551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '28551e3575b24089b0043692f65a8ffd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:54.948635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c9a0e62e-ed39-4f82-88be-ec712aaf4433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '28551e3575b24089b0043692f65a8ffd'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:55.051706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 856/1714] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:54 2022] DELETE /identity/v3/users/28551e3575b24089b0043692f65a8ffd => generated 0 bytes in 895 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:55.057221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c669e5-13d6-4fab-ae61-ce2c699e1d8c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:55.057548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c669e5-13d6-4fab-ae61-ce2c699e1d8c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:55.062938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c669e5-13d6-4fab-ae61-ce2c699e1d8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:55.085490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:55.089031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54e2e2-404b-43da-befe-d07c75e982c1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:55.091066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc54e2e2-404b-43da-befe-d07c75e982c1 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:55.137162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-69c669e5-13d6-4fab-ae61-ce2c699e1d8c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:55.139759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-69c669e5-13d6-4fab-ae61-ce2c699e1d8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:55.155686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 859/1715] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:54 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:23:55.163446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de09e1-bbcc-4712-91b1-27a6e438d416 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:55.163693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de09e1-bbcc-4712-91b1-27a6e438d416 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:55.163974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41de09e1-bbcc-4712-91b1-27a6e438d416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:55.287610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-41de09e1-bbcc-4712-91b1-27a6e438d416 None None] MFA Rules not processed for user `3a1a88089d384964bd9774eda4c5d8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:55.297003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-41de09e1-bbcc-4712-91b1-27a6e438d416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:55.446999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 857/1716] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:55.449250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad043e65-1dc1-4b17-8f8b-41eeff0f829d None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:55.460270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad043e65-1dc1-4b17-8f8b-41eeff0f829d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:55.461836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad043e65-1dc1-4b17-8f8b-41eeff0f829d None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:55.481752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 858/1717] 10.222.0.69 () {54 vars in 912 bytes} [Wed Jul 20 19:23:55 2022] GET /identity => generated 270 bytes in 33 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:23:55.498499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b29bf-70c5-4637-8f20-e045ae04c45a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:55.499172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b29bf-70c5-4637-8f20-e045ae04c45a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:55.499666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17b29bf-70c5-4637-8f20-e045ae04c45a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:55.530270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 860/1718] 10.222.0.93 () {58 vars in 1015 bytes} [Wed Jul 20 19:23:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 374 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:55.541472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de09e1-bbcc-4712-91b1-27a6e438d416 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:55.542914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-41de09e1-bbcc-4712-91b1-27a6e438d416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:55.552708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b17b29bf-70c5-4637-8f20-e045ae04c45a None None] MFA Rules not processed for user `3a1a88089d384964bd9774eda4c5d8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:55.566933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b17b29bf-70c5-4637-8f20-e045ae04c45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:55.596178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41de09e1-bbcc-4712-91b1-27a6e438d416 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:55.600993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 859/1719] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 19:23:55 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:55.606011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a5e14-da17-492d-9450-f0a23fcc4aad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:55.609187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a5e14-da17-492d-9450-f0a23fcc4aad service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:55.610335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a5e14-da17-492d-9450-f0a23fcc4aad service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:55.610637 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a5e14-da17-492d-9450-f0a23fcc4aad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:55.647853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a5e14-da17-492d-9450-f0a23fcc4aad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:55.654575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a5e14-da17-492d-9450-f0a23fcc4aad service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:55.691648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 861/1720] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:55.701366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b29bf-70c5-4637-8f20-e045ae04c45a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:55.702980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b17b29bf-70c5-4637-8f20-e045ae04c45a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:55.728737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17b29bf-70c5-4637-8f20-e045ae04c45a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:55.728961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a5e14-da17-492d-9450-f0a23fcc4aad service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:55.740725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6714e5cc-0204-4746-a2af-09feecfa2e55 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:55.742717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a5e14-da17-492d-9450-f0a23fcc4aad service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:55.744918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714e5cc-0204-4746-a2af-09feecfa2e55 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:55.745637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714e5cc-0204-4746-a2af-09feecfa2e55 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:55.747314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6714e5cc-0204-4746-a2af-09feecfa2e55 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:55.761378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997e54d-a6b9-4959-a329-ea671860c0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:55.762179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997e54d-a6b9-4959-a329-ea671860c0dd service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:55.762472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997e54d-a6b9-4959-a329-ea671860c0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:55.762756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7997e54d-a6b9-4959-a329-ea671860c0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:55.782566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714e5cc-0204-4746-a2af-09feecfa2e55 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:55.787281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6714e5cc-0204-4746-a2af-09feecfa2e55 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:55.812858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997e54d-a6b9-4959-a329-ea671860c0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:55.823730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7997e54d-a6b9-4959-a329-ea671860c0dd service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:55.829442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 860/1721] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:23:55.861545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 862/1722] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:23:55 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:23:56.608349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434f3db-2559-4319-a003-ad8c5257cca2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.610030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434f3db-2559-4319-a003-ad8c5257cca2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.610559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8434f3db-2559-4319-a003-ad8c5257cca2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.660277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8434f3db-2559-4319-a003-ad8c5257cca2 None None] MFA Rules not processed for user `9192f2842ca044f08e5f95f98f2e1b88`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:56.663880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997e54d-a6b9-4959-a329-ea671860c0dd service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:56.665060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8434f3db-2559-4319-a003-ad8c5257cca2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:56.665685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7997e54d-a6b9-4959-a329-ea671860c0dd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:56.714307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7997e54d-a6b9-4959-a329-ea671860c0dd service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:56.742776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:56.745835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:56.746134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:56.747751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] PATH_INFO: `/v3/projects/e9f126ae433048698e1504e7f55c6805` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:56.749193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f126ae433048698e1504e7f55c6805)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:56.756598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:56.858935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 861/1723] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:56.960599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f126ae433048698e1504e7f55c6805', 'request_id': 'req-1cecbdee-c1cb-4c56-ad68-326351b16200'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:56.989000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f126ae433048698e1504e7f55c6805', 'request_id': 'req-1cecbdee-c1cb-4c56-ad68-326351b16200'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:57.005044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:57.009362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f126ae433048698e1504e7f55c6805', 'request_id': 'req-1cecbdee-c1cb-4c56-ad68-326351b16200'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:57.042658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] The token cache is being invalidate because project e9f126ae433048698e1504e7f55c6805 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:23:57.043655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:57.045804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1cecbdee-c1cb-4c56-ad68-326351b16200 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:57.050777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 863/1724] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:23:56 2022] DELETE /identity/v3/projects/e9f126ae433048698e1504e7f55c6805 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:57.057981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af942181-7ac9-4095-8f0c-dbb3556262f5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.058533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af942181-7ac9-4095-8f0c-dbb3556262f5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.059880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af942181-7ac9-4095-8f0c-dbb3556262f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.110657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-af942181-7ac9-4095-8f0c-dbb3556262f5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:57.113699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-af942181-7ac9-4095-8f0c-dbb3556262f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:57.338999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 862/1725] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:57.348356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550420c6-37a4-4cf0-8421-189a90405018 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.349181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550420c6-37a4-4cf0-8421-189a90405018 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.350026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550420c6-37a4-4cf0-8421-189a90405018 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.416483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-550420c6-37a4-4cf0-8421-189a90405018 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:57.421865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-550420c6-37a4-4cf0-8421-189a90405018 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:57.452266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 864/1726] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:57.454445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af942181-7ac9-4095-8f0c-dbb3556262f5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:57.470712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af942181-7ac9-4095-8f0c-dbb3556262f5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:57.496228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555938dc-aeb0-493f-944b-356854fc8fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:57.497078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555938dc-aeb0-493f-944b-356854fc8fb2 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.497468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555938dc-aeb0-493f-944b-356854fc8fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.497873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555938dc-aeb0-493f-944b-356854fc8fb2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.499452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555938dc-aeb0-493f-944b-356854fc8fb2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:23:57.500150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555938dc-aeb0-493f-944b-356854fc8fb2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:57.504862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555938dc-aeb0-493f-944b-356854fc8fb2 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:57.541963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 863/1727] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:23:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:23:57.546144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550420c6-37a4-4cf0-8421-189a90405018 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:57.560752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550420c6-37a4-4cf0-8421-189a90405018 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:57.588917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057b166-829d-486f-84a5-6230fd3dcb84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:57.589818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057b166-829d-486f-84a5-6230fd3dcb84 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.590106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057b166-829d-486f-84a5-6230fd3dcb84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.590381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5057b166-829d-486f-84a5-6230fd3dcb84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.591193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057b166-829d-486f-84a5-6230fd3dcb84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:57.593423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5057b166-829d-486f-84a5-6230fd3dcb84 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:57.678910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 865/1728] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:57 2022] POST /identity/v3/projects => generated 373 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:57.687616 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555938dc-aeb0-493f-944b-356854fc8fb2 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:57.715130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555938dc-aeb0-493f-944b-356854fc8fb2 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:57.755415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:57.947496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bd2f1-9a93-4929-b971-8a3b1c8d7db7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:57.949619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bd2f1-9a93-4929-b971-8a3b1c8d7db7 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:57.950630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bd2f1-9a93-4929-b971-8a3b1c8d7db7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:57.952855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59bd2f1-9a93-4929-b971-8a3b1c8d7db7 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:57.957593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bd2f1-9a93-4929-b971-8a3b1c8d7db7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:57.970994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d59bd2f1-9a93-4929-b971-8a3b1c8d7db7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:58.058568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 864/1729] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:57 2022] POST /identity/v3/users => generated 287 bytes in 369 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:58.059851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057b166-829d-486f-84a5-6230fd3dcb84 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:58.088918 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5057b166-829d-486f-84a5-6230fd3dcb84 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:58.105906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9f930-7d12-4996-9edd-7edb2bbee797 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:58.109388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9f930-7d12-4996-9edd-7edb2bbee797 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:58.109928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9f930-7d12-4996-9edd-7edb2bbee797 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:58.110401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9f930-7d12-4996-9edd-7edb2bbee797 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:58.111358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9f930-7d12-4996-9edd-7edb2bbee797 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:58.113164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9f930-7d12-4996-9edd-7edb2bbee797 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:58.150175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bd2f1-9a93-4929-b971-8a3b1c8d7db7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:58.160929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 866/1730] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:58.163883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59bd2f1-9a93-4929-b971-8a3b1c8d7db7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:58.178287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e608ac-3bab-48a0-a59f-222222a8f28e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:58.179763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e608ac-3bab-48a0-a59f-222222a8f28e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:58.180503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e608ac-3bab-48a0-a59f-222222a8f28e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:58.181087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e608ac-3bab-48a0-a59f-222222a8f28e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:58.182048 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e608ac-3bab-48a0-a59f-222222a8f28e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:58.187257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e608ac-3bab-48a0-a59f-222222a8f28e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:58.216264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9f930-7d12-4996-9edd-7edb2bbee797 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:58.224676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 865/1731] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:58 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:23:58.231609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9f930-7d12-4996-9edd-7edb2bbee797 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:58.247510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554792c-ab85-42dc-87c1-86d2fde7873e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:58.249044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554792c-ab85-42dc-87c1-86d2fde7873e admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:58.249537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554792c-ab85-42dc-87c1-86d2fde7873e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:58.250238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0554792c-ab85-42dc-87c1-86d2fde7873e admin admin] PATH_INFO: `/v3/projects/ed830d9990f44c47a6311c849041a28c/users/f2337757ab4a4a21bf214744a8d0dc43/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:58.301011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554792c-ab85-42dc-87c1-86d2fde7873e admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed830d9990f44c47a6311c849041a28c, user_id=f2337757ab4a4a21bf214744a8d0dc43, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:58.303889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0554792c-ab85-42dc-87c1-86d2fde7873e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:58.351998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 867/1732] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:58 2022] PUT /identity/v3/projects/ed830d9990f44c47a6311c849041a28c/users/f2337757ab4a4a21bf214744a8d0dc43/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:58.364989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c8db-cbc7-4fa9-b083-2b563850e84d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:58.364989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c8db-cbc7-4fa9-b083-2b563850e84d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:58.365280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a16c8db-cbc7-4fa9-b083-2b563850e84d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:58.446044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-9a16c8db-cbc7-4fa9-b083-2b563850e84d None None] MFA Rules not processed for user `f2337757ab4a4a21bf214744a8d0dc43`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:23:58.471319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9a16c8db-cbc7-4fa9-b083-2b563850e84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:58.755197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 866/1733] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:23:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:23:58.763534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554792c-ab85-42dc-87c1-86d2fde7873e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:59.016074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c8db-cbc7-4fa9-b083-2b563850e84d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:59.017908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9a16c8db-cbc7-4fa9-b083-2b563850e84d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:23:59.087650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0554792c-ab85-42dc-87c1-86d2fde7873e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:59.114014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb224c6-f320-49c6-93e1-dbbc99682a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:59.116078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb224c6-f320-49c6-93e1-dbbc99682a1c admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:59.117044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb224c6-f320-49c6-93e1-dbbc99682a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:59.125895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb224c6-f320-49c6-93e1-dbbc99682a1c admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:59.129177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb224c6-f320-49c6-93e1-dbbc99682a1c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:59.135316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb224c6-f320-49c6-93e1-dbbc99682a1c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:59.321324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 868/1734] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:23:58 2022] POST /identity/v3/projects => generated 373 bytes in 559 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:59.330130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb224c6-f320-49c6-93e1-dbbc99682a1c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:59.331636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a16c8db-cbc7-4fa9-b083-2b563850e84d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:59.549396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:59.558447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb224c6-f320-49c6-93e1-dbbc99682a1c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:59.572977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa54115-5067-44bb-9bdf-d3de1e85806e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:59.573721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54115-5067-44bb-9bdf-d3de1e85806e admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:59.574008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54115-5067-44bb-9bdf-d3de1e85806e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:59.574285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa54115-5067-44bb-9bdf-d3de1e85806e admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:59.574983 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa54115-5067-44bb-9bdf-d3de1e85806e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:59.577095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa54115-5067-44bb-9bdf-d3de1e85806e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:59.619749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=2,keystone:SELECT=91,keystone:INSERT=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:23:59.647952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 869/1735] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:23:59 2022] POST /identity/v3/users => generated 293 bytes in 323 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:23:59.653762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa54115-5067-44bb-9bdf-d3de1e85806e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:59.689996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa54115-5067-44bb-9bdf-d3de1e85806e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:59.692091 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:59.695797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:59.696367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:59.697978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] PATH_INFO: `/v3/users/62d7396dc47a4c48a4e581872da239b8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:59.703682 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62d7396dc47a4c48a4e581872da239b8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:59.714369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:59.738981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c763f62-f9b7-4160-8b45-3add0f8c60c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:59.740708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c763f62-f9b7-4160-8b45-3add0f8c60c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:59.741188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c763f62-f9b7-4160-8b45-3add0f8c60c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:59.742399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c763f62-f9b7-4160-8b45-3add0f8c60c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:59.743346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c763f62-f9b7-4160-8b45-3add0f8c60c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:59.745119 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c763f62-f9b7-4160-8b45-3add0f8c60c8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:59.773917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 870/1736] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:23:59.776069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c763f62-f9b7-4160-8b45-3add0f8c60c8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:59.815789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c763f62-f9b7-4160-8b45-3add0f8c60c8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:23:59.834311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69e406-612f-463e-89bd-14a60efd2d29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:23:59.836132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69e406-612f-463e-89bd-14a60efd2d29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:23:59.837532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69e406-612f-463e-89bd-14a60efd2d29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:23:59.838176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a69e406-612f-463e-89bd-14a60efd2d29 admin admin] PATH_INFO: `/v3/projects/3e2795852ec64b14b6f52369f005c638/users/588df8c6b7294e3a8f5e1a2f289d2a58/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:23:59.880628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69e406-612f-463e-89bd-14a60efd2d29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2795852ec64b14b6f52369f005c638, user_id=588df8c6b7294e3a8f5e1a2f289d2a58, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:23:59.896529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a69e406-612f-463e-89bd-14a60efd2d29 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:23:59.924507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62d7396dc47a4c48a4e581872da239b8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:23:59.927444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 871/1737] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:23:59 2022] PUT /identity/v3/projects/3e2795852ec64b14b6f52369f005c638/users/588df8c6b7294e3a8f5e1a2f289d2a58/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:23:59.933337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69e406-612f-463e-89bd-14a60efd2d29 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:23:59.998112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62d7396dc47a4c48a4e581872da239b8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.065822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62d7396dc47a4c48a4e581872da239b8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.078653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62d7396dc47a4c48a4e581872da239b8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.083942 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62d7396dc47a4c48a4e581872da239b8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.162343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 867/1738] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:23:59 2022] DELETE /identity/v3/users/62d7396dc47a4c48a4e581872da239b8 => generated 0 bytes in 1143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:00.162833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:00.178307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:00.302673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a69e406-612f-463e-89bd-14a60efd2d29 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:00.445549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa0091f-3732-4e1f-a550-6397c2ae7cb7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:00.459351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:00.461261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.462070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.463054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] PATH_INFO: `/v3/users/47478871f08e4c58be083739eb1ee56b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.466589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47478871f08e4c58be083739eb1ee56b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:00.469619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:00.500814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e0675-f504-4f40-8104-35e7e4eed72b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:00.501874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e0675-f504-4f40-8104-35e7e4eed72b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.502387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e0675-f504-4f40-8104-35e7e4eed72b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.502872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954e0675-f504-4f40-8104-35e7e4eed72b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.503830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e0675-f504-4f40-8104-35e7e4eed72b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:00.505753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954e0675-f504-4f40-8104-35e7e4eed72b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:00.532108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 872/1739] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:23:59 2022] GET /identity/v3/roles => generated 1420 bytes in 599 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:00.539496 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e0675-f504-4f40-8104-35e7e4eed72b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:00.543097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-954e0675-f504-4f40-8104-35e7e4eed72b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:00.600709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47478871f08e4c58be083739eb1ee56b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.609481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:00.627178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47478871f08e4c58be083739eb1ee56b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.630584 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954e0675-f504-4f40-8104-35e7e4eed72b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:00.642772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47478871f08e4c58be083739eb1ee56b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.646986 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47478871f08e4c58be083739eb1ee56b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.648829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-296f78bf-c19e-4af8-82ba-38c6df2bf163 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47478871f08e4c58be083739eb1ee56b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.658570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:00.666738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.667746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.668750 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] PATH_INFO: `/v3/projects/0dd360813c734ddfa4e36fd90acfcb1c` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.681536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dd360813c734ddfa4e36fd90acfcb1c)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:00.691796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:00.699752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 868/1740] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:00 2022] DELETE /identity/v3/users/47478871f08e4c58be083739eb1ee56b => generated 0 bytes in 537 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:00.714602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedd07b-ff6c-4fb4-9b3b-d9e77406c482 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:00.715600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedd07b-ff6c-4fb4-9b3b-d9e77406c482 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:00.716549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cedd07b-ff6c-4fb4-9b3b-d9e77406c482 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:00.775436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2cedd07b-ff6c-4fb4-9b3b-d9e77406c482 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:00.777638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2cedd07b-ff6c-4fb4-9b3b-d9e77406c482 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:00.881826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dd360813c734ddfa4e36fd90acfcb1c', 'request_id': 'req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.922135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dd360813c734ddfa4e36fd90acfcb1c', 'request_id': 'req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:00.944419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:INSERT=1,keystone:DELETE=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:00.950586 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dd360813c734ddfa4e36fd90acfcb1c', 'request_id': 'req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.004624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] The token cache is being invalidate because project 0dd360813c734ddfa4e36fd90acfcb1c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:01.005927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.009841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4fd726e3-a2cc-43ad-af36-db34d1bbd04a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.019561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 873/1741] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:00 2022] DELETE /identity/v3/projects/0dd360813c734ddfa4e36fd90acfcb1c => generated 0 bytes in 479 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:01.019561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb575-afc0-419f-9877-949c297b9f7f None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.027882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb575-afc0-419f-9877-949c297b9f7f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.030297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cccb575-afc0-419f-9877-949c297b9f7f None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.042241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 874/1742] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:24:01 2022] GET /identity => generated 270 bytes in 25 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:24:01.043268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccb575-afc0-419f-9877-949c297b9f7f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:01.049668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1cccb575-afc0-419f-9877-949c297b9f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:01.054685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 869/1743] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:01.062729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52439354-df34-49ba-b086-0fc3c252f939 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.063401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52439354-df34-49ba-b086-0fc3c252f939 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.064123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52439354-df34-49ba-b086-0fc3c252f939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.104432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-52439354-df34-49ba-b086-0fc3c252f939 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:01.107379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-52439354-df34-49ba-b086-0fc3c252f939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:01.245665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cccb575-afc0-419f-9877-949c297b9f7f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:01.249671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 870/1744] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:01.254054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8a8f-fd1c-43ac-b915-d6d56c2e4398 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.254963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8a8f-fd1c-43ac-b915-d6d56c2e4398 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.255280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ed8a8f-fd1c-43ac-b915-d6d56c2e4398 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.258356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d391e-e33b-46e2-804f-dfedba49c704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:01.259498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d391e-e33b-46e2-804f-dfedba49c704 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.264224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d391e-e33b-46e2-804f-dfedba49c704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.264955 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5d391e-e33b-46e2-804f-dfedba49c704 admin admin] PATH_INFO: `/v3/projects/3e2795852ec64b14b6f52369f005c638/users/588df8c6b7294e3a8f5e1a2f289d2a58/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.270784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d391e-e33b-46e2-804f-dfedba49c704 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e2795852ec64b14b6f52369f005c638, user_id=588df8c6b7294e3a8f5e1a2f289d2a58, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:01.274622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5d391e-e33b-46e2-804f-dfedba49c704 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:01.286084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-25ed8a8f-fd1c-43ac-b915-d6d56c2e4398 None None] MFA Rules not processed for user `4d08da0ef69e4cf2abc6c18bbbc51a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:01.287846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-25ed8a8f-fd1c-43ac-b915-d6d56c2e4398 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:01.307536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 875/1745] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:01 2022] PUT /identity/v3/projects/3e2795852ec64b14b6f52369f005c638/users/588df8c6b7294e3a8f5e1a2f289d2a58/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:01.308733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d391e-e33b-46e2-804f-dfedba49c704 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:01.325764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0c5d391e-e33b-46e2-804f-dfedba49c704 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:01.401803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 871/1746] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:01.403806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7224b90-25b4-4724-af19-869669f3b962 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.404131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7224b90-25b4-4724-af19-869669f3b962 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.404412 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7224b90-25b4-4724-af19-869669f3b962 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.434926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e7224b90-25b4-4724-af19-869669f3b962 None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:01.436181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e7224b90-25b4-4724-af19-869669f3b962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:01.454486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:01.607188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5d391e-e33b-46e2-804f-dfedba49c704 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:01.621595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:01.622754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.623077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.624111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] PATH_INFO: `/v3/users/72d02479b2b94506be98d917a3964880` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.630460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] RBAC: Authorizing `identity:delete_user(user_id=72d02479b2b94506be98d917a3964880)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:01.631572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: WARNING py.warnings [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:24:01.631572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: warnings.warn(msg) Jul 20 19:24:01.631572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: Jul 20 19:24:01.633834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:01.642769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 872/1747] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:24:01 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:01.645143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7224b90-25b4-4724-af19-869669f3b962 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:01.647101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e7224b90-25b4-4724-af19-869669f3b962 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:01.789326 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7224b90-25b4-4724-af19-869669f3b962 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:01.795829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f93fba-5bbe-450d-99bc-4ac8777644f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:01.796856 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93fba-5bbe-450d-99bc-4ac8777644f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.798044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93fba-5bbe-450d-99bc-4ac8777644f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.798587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f93fba-5bbe-450d-99bc-4ac8777644f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.827081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f93fba-5bbe-450d-99bc-4ac8777644f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:01.833377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f93fba-5bbe-450d-99bc-4ac8777644f6 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:01.833874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72d02479b2b94506be98d917a3964880'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.868229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 873/1748] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:01.870678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49880caa-ed27-4551-8877-dad638a653ba None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:01.870969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49880caa-ed27-4551-8877-dad638a653ba None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:01.873138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49880caa-ed27-4551-8877-dad638a653ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:01.894949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72d02479b2b94506be98d917a3964880'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.948896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72d02479b2b94506be98d917a3964880'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.952330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72d02479b2b94506be98d917a3964880'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.956062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72d02479b2b94506be98d917a3964880'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:01.992989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-49880caa-ed27-4551-8877-dad638a653ba None None] MFA Rules not processed for user `588df8c6b7294e3a8f5e1a2f289d2a58`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:01.999344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-49880caa-ed27-4551-8877-dad638a653ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.001468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 876/1749] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:01 2022] DELETE /identity/v3/users/72d02479b2b94506be98d917a3964880 => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:02.011169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:02.018767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.044455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=86,keystone:INSERT=3,keystone:DELETE=11 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:02.212977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f63e08e-4f76-4ef1-8333-6affeb74cecd admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:02.218996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5490e25-c302-4da8-abc9-13b82a9f29fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:02.225074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5490e25-c302-4da8-abc9-13b82a9f29fc service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:02.225367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5490e25-c302-4da8-abc9-13b82a9f29fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:02.225823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5490e25-c302-4da8-abc9-13b82a9f29fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:02.229210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a5490e25-c302-4da8-abc9-13b82a9f29fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.292715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 874/1750] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:01 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 421 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:02.300696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49880caa-ed27-4551-8877-dad638a653ba None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:02.307672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-49880caa-ed27-4551-8877-dad638a653ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.398434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5490e25-c302-4da8-abc9-13b82a9f29fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:02.400588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5490e25-c302-4da8-abc9-13b82a9f29fc service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:02.417278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 877/1751] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:02.419347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5490e25-c302-4da8-abc9-13b82a9f29fc service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:02.421281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a5490e25-c302-4da8-abc9-13b82a9f29fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.432294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:02.585770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5490e25-c302-4da8-abc9-13b82a9f29fc service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:02.590402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769daa2-2587-42c1-a887-6fe31d303201 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:02.591119 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769daa2-2587-42c1-a887-6fe31d303201 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:02.591474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769daa2-2587-42c1-a887-6fe31d303201 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:02.591776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4769daa2-2587-42c1-a887-6fe31d303201 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:02.625007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769daa2-2587-42c1-a887-6fe31d303201 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:02.630748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4769daa2-2587-42c1-a887-6fe31d303201 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:02.657253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49880caa-ed27-4551-8877-dad638a653ba None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:02.667787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883f788a-67b7-4d3a-b2f2-f8bf82a6a119 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:02.668765 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f788a-67b7-4d3a-b2f2-f8bf82a6a119 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:02.669172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f788a-67b7-4d3a-b2f2-f8bf82a6a119 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:02.669597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883f788a-67b7-4d3a-b2f2-f8bf82a6a119 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:02.672409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-883f788a-67b7-4d3a-b2f2-f8bf82a6a119 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:02.726651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 878/1752] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:02 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:02.730930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f788a-67b7-4d3a-b2f2-f8bf82a6a119 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:02.744586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883f788a-67b7-4d3a-b2f2-f8bf82a6a119 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:02.792617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 875/1753] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 500 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:03.859545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769daa2-2587-42c1-a887-6fe31d303201 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:03.863177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4769daa2-2587-42c1-a887-6fe31d303201 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:03.897846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4769daa2-2587-42c1-a887-6fe31d303201 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:03.920526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:03.920861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:03.921158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:03.921437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] PATH_INFO: `/v3/users/f2337757ab4a4a21bf214744a8d0dc43` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:03.923278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2337757ab4a4a21bf214744a8d0dc43)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:03.925299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:04.041016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2337757ab4a4a21bf214744a8d0dc43'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:04.047354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:04.080186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2337757ab4a4a21bf214744a8d0dc43'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:04.100137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2337757ab4a4a21bf214744a8d0dc43'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:04.101962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2337757ab4a4a21bf214744a8d0dc43'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:04.103131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2337757ab4a4a21bf214744a8d0dc43'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:04.130693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 879/1754] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:03 2022] DELETE /identity/v3/users/f2337757ab4a4a21bf214744a8d0dc43 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:04.132723 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef4701-16d1-4698-91f9-79e39132b0c9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:04.133388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef4701-16d1-4698-91f9-79e39132b0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:04.133490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ef4701-16d1-4698-91f9-79e39132b0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:04.177152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a9ef4701-16d1-4698-91f9-79e39132b0c9 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:04.178464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a9ef4701-16d1-4698-91f9-79e39132b0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:04.355685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 876/1755] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:04.429099 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:04.431268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:04.654125 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc978736-dc55-4277-973d-2833c4843c25 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:04.662272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e6e0c9-0a70-4a30-a00f-be3c77b658cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:04.663006 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6e0c9-0a70-4a30-a00f-be3c77b658cc service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:04.663480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6e0c9-0a70-4a30-a00f-be3c77b658cc service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:04.666726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e6e0c9-0a70-4a30-a00f-be3c77b658cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:04.689390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6e0c9-0a70-4a30-a00f-be3c77b658cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:04.691880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e6e0c9-0a70-4a30-a00f-be3c77b658cc service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:04.724727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 880/1756] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:06.321729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef4701-16d1-4698-91f9-79e39132b0c9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:06.323223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a9ef4701-16d1-4698-91f9-79e39132b0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:06.340216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9ef4701-16d1-4698-91f9-79e39132b0c9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:06.390433 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:06.391369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:06.391658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:06.392008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] PATH_INFO: `/v3/projects/f97d002af3ee402090e62737cf1e14b9` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:06.399877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=f97d002af3ee402090e62737cf1e14b9)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:06.400971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: WARNING py.warnings [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 20 19:24:06.400971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: warnings.warn(msg) Jul 20 19:24:06.400971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: Jul 20 19:24:06.402313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:06.611954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f97d002af3ee402090e62737cf1e14b9', 'request_id': 'req-da175f13-d446-43e3-9ff8-8adf2e989f3d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:06.621978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f97d002af3ee402090e62737cf1e14b9', 'request_id': 'req-da175f13-d446-43e3-9ff8-8adf2e989f3d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:06.637675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f97d002af3ee402090e62737cf1e14b9', 'request_id': 'req-da175f13-d446-43e3-9ff8-8adf2e989f3d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:06.680729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] The token cache is being invalidate because project f97d002af3ee402090e62737cf1e14b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:06.681065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:06.681826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:06.687917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 877/1757] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:06 2022] DELETE /identity/v3/projects/f97d002af3ee402090e62737cf1e14b9 => generated 0 bytes in 362 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:06.690105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343157cd-ae39-475e-b19b-ab80a5bae51e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:06.690866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343157cd-ae39-475e-b19b-ab80a5bae51e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:06.690866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343157cd-ae39-475e-b19b-ab80a5bae51e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:06.724178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-343157cd-ae39-475e-b19b-ab80a5bae51e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:06.726838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-343157cd-ae39-475e-b19b-ab80a5bae51e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:06.799326 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:06.801155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:06.935639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 881/1758] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:06.943935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56137f6-bbf8-4e1c-9b82-93c6f1061596 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:06.945003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56137f6-bbf8-4e1c-9b82-93c6f1061596 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:06.946514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e56137f6-bbf8-4e1c-9b82-93c6f1061596 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:06.947271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:07.004620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e56137f6-bbf8-4e1c-9b82-93c6f1061596 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:07.008901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da175f13-d446-43e3-9ff8-8adf2e989f3d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:07.011292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e56137f6-bbf8-4e1c-9b82-93c6f1061596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:07.017849 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:07.019329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.020161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.020978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] PATH_INFO: `/v3/projects/ff8cb4a4fb48431db60b8b5e89c51ea1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:07.025022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff8cb4a4fb48431db60b8b5e89c51ea1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:07.027254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:07.037003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 882/1759] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:07.039110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56137f6-bbf8-4e1c-9b82-93c6f1061596 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:07.066191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e56137f6-bbf8-4e1c-9b82-93c6f1061596 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:07.080969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1fdfa-69e9-49dc-b8ed-af4659d73464 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:07.082003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1fdfa-69e9-49dc-b8ed-af4659d73464 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.082685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1fdfa-69e9-49dc-b8ed-af4659d73464 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.082989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e1fdfa-69e9-49dc-b8ed-af4659d73464 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:07.084353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1fdfa-69e9-49dc-b8ed-af4659d73464 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:07.087202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1fdfa-69e9-49dc-b8ed-af4659d73464 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:07.089202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e1fdfa-69e9-49dc-b8ed-af4659d73464 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:07.117573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 883/1760] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:07.139959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1fdfa-69e9-49dc-b8ed-af4659d73464 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:07.174392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e1fdfa-69e9-49dc-b8ed-af4659d73464 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:07.191392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff8cb4a4fb48431db60b8b5e89c51ea1', 'request_id': 'req-33b55508-fd73-493a-8130-e125d7d4e6f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:07.227338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff8cb4a4fb48431db60b8b5e89c51ea1', 'request_id': 'req-33b55508-fd73-493a-8130-e125d7d4e6f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:07.245814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff8cb4a4fb48431db60b8b5e89c51ea1', 'request_id': 'req-33b55508-fd73-493a-8130-e125d7d4e6f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:07.270974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] The token cache is being invalidate because project ff8cb4a4fb48431db60b8b5e89c51ea1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:07.273423 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:07.275690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-33b55508-fd73-493a-8130-e125d7d4e6f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:07.286812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 878/1761] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:06 2022] DELETE /identity/v3/projects/ff8cb4a4fb48431db60b8b5e89c51ea1 => generated 0 bytes in 485 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:07.290949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa034-b485-4245-8dc3-e09c90420ffc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.291338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa034-b485-4245-8dc3-e09c90420ffc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.292029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccafa034-b485-4245-8dc3-e09c90420ffc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:07.326974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ccafa034-b485-4245-8dc3-e09c90420ffc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:07.341133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ccafa034-b485-4245-8dc3-e09c90420ffc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:07.349264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:07.398194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090d610-d41c-42db-8d88-0fbe228ef339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:07.398194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090d610-d41c-42db-8d88-0fbe228ef339 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.398194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090d610-d41c-42db-8d88-0fbe228ef339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.398194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0090d610-d41c-42db-8d88-0fbe228ef339 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:07.399621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090d610-d41c-42db-8d88-0fbe228ef339 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:07.399621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0090d610-d41c-42db-8d88-0fbe228ef339 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:07.462672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 884/1762] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:07 2022] POST /identity/v3/projects => generated 387 bytes in 327 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:07.463654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090d610-d41c-42db-8d88-0fbe228ef339 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:07.465871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0090d610-d41c-42db-8d88-0fbe228ef339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:07.550629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 879/1763] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:07.550629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c36b7e-6c8d-4b5d-a31b-600044118008 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.550629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c36b7e-6c8d-4b5d-a31b-600044118008 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.550629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c36b7e-6c8d-4b5d-a31b-600044118008 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:07.560155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 880/1764] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:24:07 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:24:07.564111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c36b7e-6c8d-4b5d-a31b-600044118008 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:07.569185 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f8c36b7e-6c8d-4b5d-a31b-600044118008 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:07.686612 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0090d610-d41c-42db-8d88-0fbe228ef339 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:07.715166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:07.716477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.716994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.717740 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] PATH_INFO: `/v3/projects/cc30e4dd5d1f4829b68718ef10261ac6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:07.719982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc30e4dd5d1f4829b68718ef10261ac6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:07.722114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:07.782503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c36b7e-6c8d-4b5d-a31b-600044118008 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:07.793381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252e5523-fe5f-4117-a6cd-9302a8cdcf5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:07.794467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e5523-fe5f-4117-a6cd-9302a8cdcf5d admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.795001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e5523-fe5f-4117-a6cd-9302a8cdcf5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.795443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252e5523-fe5f-4117-a6cd-9302a8cdcf5d admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:07.796248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e5523-fe5f-4117-a6cd-9302a8cdcf5d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:07.805657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252e5523-fe5f-4117-a6cd-9302a8cdcf5d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:07.849230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc30e4dd5d1f4829b68718ef10261ac6', 'request_id': 'req-01db726a-a5bf-4ac5-9202-23183cbc6ac1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:07.857608 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 881/1765] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:07 2022] POST /identity/v3/users => generated 294 bytes in 296 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:07.862641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52823c71-2b4f-4a58-a1d6-e709c4c9a668 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.862884 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52823c71-2b4f-4a58-a1d6-e709c4c9a668 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.863369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52823c71-2b4f-4a58-a1d6-e709c4c9a668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:07.867992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc30e4dd5d1f4829b68718ef10261ac6', 'request_id': 'req-01db726a-a5bf-4ac5-9202-23183cbc6ac1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:07.880888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:07.884171 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc30e4dd5d1f4829b68718ef10261ac6', 'request_id': 'req-01db726a-a5bf-4ac5-9202-23183cbc6ac1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:07.937206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] The token cache is being invalidate because project cc30e4dd5d1f4829b68718ef10261ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:07.937902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:07.938896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-01db726a-a5bf-4ac5-9202-23183cbc6ac1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:07.943702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 885/1766] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:07 2022] DELETE /identity/v3/projects/cc30e4dd5d1f4829b68718ef10261ac6 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:07.950556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-52823c71-2b4f-4a58-a1d6-e709c4c9a668 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:07.951169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91b363-1b40-4aa5-9dff-310e0ae10b5d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:07.952118 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91b363-1b40-4aa5-9dff-310e0ae10b5d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:07.954186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-52823c71-2b4f-4a58-a1d6-e709c4c9a668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:07.954805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb91b363-1b40-4aa5-9dff-310e0ae10b5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.057705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bb91b363-1b40-4aa5-9dff-310e0ae10b5d None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:08.059337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bb91b363-1b40-4aa5-9dff-310e0ae10b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:08.275438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:08.301971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 882/1767] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 444 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:08.303159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52823c71-2b4f-4a58-a1d6-e709c4c9a668 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.305913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-52823c71-2b4f-4a58-a1d6-e709c4c9a668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:08.325949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 886/1768] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:24:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 381 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:08.327728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91b363-1b40-4aa5-9dff-310e0ae10b5d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.334945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bb91b363-1b40-4aa5-9dff-310e0ae10b5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:08.347985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52823c71-2b4f-4a58-a1d6-e709c4c9a668 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:08.368018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb91b363-1b40-4aa5-9dff-310e0ae10b5d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:08.371215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b0ec0-299f-4869-99ec-7d429b4a8170 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:08.372213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b0ec0-299f-4869-99ec-7d429b4a8170 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:08.373232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b0ec0-299f-4869-99ec-7d429b4a8170 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:08.373752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e0b0ec0-299f-4869-99ec-7d429b4a8170 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.377277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b0ec0-299f-4869-99ec-7d429b4a8170 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:08.381240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e0b0ec0-299f-4869-99ec-7d429b4a8170 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:08.383069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:08.384261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:08.385092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:08.385461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] PATH_INFO: `/v3/users/0d26c40913834ebf943708ab23f9cfda` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.388956 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d26c40913834ebf943708ab23f9cfda)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:08.391009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:08.418971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 883/1769] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:08.420602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b0ec0-299f-4869-99ec-7d429b4a8170 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.450369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e0b0ec0-299f-4869-99ec-7d429b4a8170 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:08.456517 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc449b1-33ac-4d15-9546-e7dd5d729892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:08.472746 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc449b1-33ac-4d15-9546-e7dd5d729892 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:08.473149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc449b1-33ac-4d15-9546-e7dd5d729892 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:08.473554 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc449b1-33ac-4d15-9546-e7dd5d729892 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.474379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc449b1-33ac-4d15-9546-e7dd5d729892 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:08.474831 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc449b1-33ac-4d15-9546-e7dd5d729892 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:08.478316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cc449b1-33ac-4d15-9546-e7dd5d729892 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:08.504229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 884/1770] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:08.505223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc449b1-33ac-4d15-9546-e7dd5d729892 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.513279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d26c40913834ebf943708ab23f9cfda'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:08.562482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d26c40913834ebf943708ab23f9cfda'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:08.591465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d26c40913834ebf943708ab23f9cfda'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:08.591465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d26c40913834ebf943708ab23f9cfda'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:08.591890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d26c40913834ebf943708ab23f9cfda'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:08.619173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc449b1-33ac-4d15-9546-e7dd5d729892 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:08.628008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 887/1771] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:08 2022] DELETE /identity/v3/users/0d26c40913834ebf943708ab23f9cfda => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:08.628864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.644529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:08.658544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec82824-894f-4082-b081-6602accd8b88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:08.660042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec82824-894f-4082-b081-6602accd8b88 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:08.660631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec82824-894f-4082-b081-6602accd8b88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:08.675926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec82824-894f-4082-b081-6602accd8b88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.676889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec82824-894f-4082-b081-6602accd8b88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:08.685831 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec82824-894f-4082-b081-6602accd8b88 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:08.747935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 885/1772] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:08 2022] GET /identity/v3/roles => generated 1420 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:08.749083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec82824-894f-4082-b081-6602accd8b88 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.753536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-eec82824-894f-4082-b081-6602accd8b88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:08.791839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec82824-894f-4082-b081-6602accd8b88 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:08.876721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51bf5b8c-c823-4b9b-83d3-028dc3b84133 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:08.900810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b710f8-553d-4ebe-a1af-8181f78523ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:08.901549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b710f8-553d-4ebe-a1af-8181f78523ab admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:08.901844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b710f8-553d-4ebe-a1af-8181f78523ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:08.902129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b710f8-553d-4ebe-a1af-8181f78523ab admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.902835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b710f8-553d-4ebe-a1af-8181f78523ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:08.904890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b710f8-553d-4ebe-a1af-8181f78523ab admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:08.909311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7740ab1-e6e1-4d52-ae77-52c1c5d272c4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:08.910560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740ab1-e6e1-4d52-ae77-52c1c5d272c4 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:08.911068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740ab1-e6e1-4d52-ae77-52c1c5d272c4 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:08.911571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7740ab1-e6e1-4d52-ae77-52c1c5d272c4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:08.915152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f7740ab1-e6e1-4d52-ae77-52c1c5d272c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:08.967015 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7740ab1-e6e1-4d52-ae77-52c1c5d272c4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:08.969594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7740ab1-e6e1-4d52-ae77-52c1c5d272c4 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:08.982274 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 888/1773] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:08 2022] POST /identity/v3/projects => generated 363 bytes in 354 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:08.982274 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b710f8-553d-4ebe-a1af-8181f78523ab admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:08.995637 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-55b710f8-553d-4ebe-a1af-8181f78523ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:09.027386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 886/1774] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:08 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 278 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:09.031326 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7740ab1-e6e1-4d52-ae77-52c1c5d272c4 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:09.033790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f7740ab1-e6e1-4d52-ae77-52c1c5d272c4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:09.113447 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7740ab1-e6e1-4d52-ae77-52c1c5d272c4 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:09.143397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:09.157278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b710f8-553d-4ebe-a1af-8181f78523ab admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:09.167186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:09.168089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:09.168372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:09.168651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] PATH_INFO: `/v3/projects/ed830d9990f44c47a6311c849041a28c` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:09.179179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed830d9990f44c47a6311c849041a28c)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:09.181211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:09.289859 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:09.305014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509620e-efdf-43a4-a8c4-ca89e3c8cf05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:09.306021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509620e-efdf-43a4-a8c4-ca89e3c8cf05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:09.306543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509620e-efdf-43a4-a8c4-ca89e3c8cf05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:09.306828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d509620e-efdf-43a4-a8c4-ca89e3c8cf05 admin admin] PATH_INFO: `/v3/projects/6ecad7bc6ee1494dbfa7c9c5fa47c97b/users/6c3ae159bf70463188a39d4fcdc244a0/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:09.308234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed830d9990f44c47a6311c849041a28c', 'request_id': 'req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:09.326602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed830d9990f44c47a6311c849041a28c', 'request_id': 'req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:09.342954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509620e-efdf-43a4-a8c4-ca89e3c8cf05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6ecad7bc6ee1494dbfa7c9c5fa47c97b, user_id=6c3ae159bf70463188a39d4fcdc244a0, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:09.347393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d509620e-efdf-43a4-a8c4-ca89e3c8cf05 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:09.348287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed830d9990f44c47a6311c849041a28c', 'request_id': 'req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:09.364644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] The token cache is being invalidate because project ed830d9990f44c47a6311c849041a28c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:09.364980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:09.370008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:09.377084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 889/1775] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:08 2022] DELETE /identity/v3/projects/ed830d9990f44c47a6311c849041a28c => generated 0 bytes in 391 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:09.378075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:09.381389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:09.382614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 887/1776] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:09 2022] PUT /identity/v3/projects/6ecad7bc6ee1494dbfa7c9c5fa47c97b/users/6c3ae159bf70463188a39d4fcdc244a0/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:09.384327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509620e-efdf-43a4-a8c4-ca89e3c8cf05 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:09.395881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d509620e-efdf-43a4-a8c4-ca89e3c8cf05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:09.494563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c7c72e-a2e5-47be-bcc8-d5f1b0cebeb6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:09.588577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddc47a2c-952b-410b-93e2-c68cb58d17f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:09.589962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc47a2c-952b-410b-93e2-c68cb58d17f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:09.590252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc47a2c-952b-410b-93e2-c68cb58d17f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:09.590532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddc47a2c-952b-410b-93e2-c68cb58d17f5 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:09.591231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc47a2c-952b-410b-93e2-c68cb58d17f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:09.595280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddc47a2c-952b-410b-93e2-c68cb58d17f5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:09.636441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 890/1777] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:09 2022] POST /identity/v3/users => generated 282 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:09.638035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7beed7-240e-4a35-b755-a10470537069 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:09.638331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7beed7-240e-4a35-b755-a10470537069 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:09.638699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c7beed7-240e-4a35-b755-a10470537069 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:09.665108 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d509620e-efdf-43a4-a8c4-ca89e3c8cf05 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:09.680897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:09.681960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:09.682253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:09.682539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] PATH_INFO: `/v3/users/588df8c6b7294e3a8f5e1a2f289d2a58` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:09.686448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] RBAC: Authorizing `identity:delete_user(user_id=588df8c6b7294e3a8f5e1a2f289d2a58)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:09.687941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3c7beed7-240e-4a35-b755-a10470537069 None None] MFA Rules not processed for user `6c3ae159bf70463188a39d4fcdc244a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:09.688368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:09.690118 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3c7beed7-240e-4a35-b755-a10470537069 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:09.705640 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:09.776603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '588df8c6b7294e3a8f5e1a2f289d2a58'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:09.831616 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '588df8c6b7294e3a8f5e1a2f289d2a58'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:09.849453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '588df8c6b7294e3a8f5e1a2f289d2a58'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:09.853116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '588df8c6b7294e3a8f5e1a2f289d2a58'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:09.855478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '588df8c6b7294e3a8f5e1a2f289d2a58'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:09.858734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 891/1778] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 222 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:09.860202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7beed7-240e-4a35-b755-a10470537069 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:09.876932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c7beed7-240e-4a35-b755-a10470537069 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:09.884870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 888/1779] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:09 2022] DELETE /identity/v3/users/588df8c6b7294e3a8f5e1a2f289d2a58 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:09.885547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:09.885999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c3b58-eb68-4d04-92b0-f2e34070f07e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:09.887795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:09.901007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c3b58-eb68-4d04-92b0-f2e34070f07e admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:09.901651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c3b58-eb68-4d04-92b0-f2e34070f07e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:09.902135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7c3b58-eb68-4d04-92b0-f2e34070f07e admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:09.903060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c3b58-eb68-4d04-92b0-f2e34070f07e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:09.909089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7c3b58-eb68-4d04-92b0-f2e34070f07e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:09.933485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 892/1780] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:09.949881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c3b58-eb68-4d04-92b0-f2e34070f07e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:09.960351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2c7c3b58-eb68-4d04-92b0-f2e34070f07e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:10.004318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7c3b58-eb68-4d04-92b0-f2e34070f07e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.063521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:10.072466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac88ce0-f8ea-45ba-bce9-5645a00a518f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.089097 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac980ef-a370-4844-abfb-ed657ba6f4e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.090917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac980ef-a370-4844-abfb-ed657ba6f4e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.091209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac980ef-a370-4844-abfb-ed657ba6f4e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.092004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac980ef-a370-4844-abfb-ed657ba6f4e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.092636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac980ef-a370-4844-abfb-ed657ba6f4e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.094781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac980ef-a370-4844-abfb-ed657ba6f4e7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.095057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406147-12d8-4c67-95c7-43ed7a59abc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.095771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406147-12d8-4c67-95c7-43ed7a59abc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.096058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406147-12d8-4c67-95c7-43ed7a59abc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.096339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b406147-12d8-4c67-95c7-43ed7a59abc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.097227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406147-12d8-4c67-95c7-43ed7a59abc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.098925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b406147-12d8-4c67-95c7-43ed7a59abc8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.113649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 893/1781] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:09 2022] GET /identity/v3/roles => generated 1420 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:10.119347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406147-12d8-4c67-95c7-43ed7a59abc8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:10.128890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 889/1782] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:09 2022] POST /identity/v3/projects => generated 385 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:10.132845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac980ef-a370-4844-abfb-ed657ba6f4e7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:10.134518 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b406147-12d8-4c67-95c7-43ed7a59abc8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.230752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac980ef-a370-4844-abfb-ed657ba6f4e7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.238050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece64a4-cfdb-4824-814d-978cbdebd2ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.238914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece64a4-cfdb-4824-814d-978cbdebd2ee admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.239204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece64a4-cfdb-4824-814d-978cbdebd2ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.239484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aece64a4-cfdb-4824-814d-978cbdebd2ee admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.240269 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece64a4-cfdb-4824-814d-978cbdebd2ee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.244052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aece64a4-cfdb-4824-814d-978cbdebd2ee admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.271916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 890/1783] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:10 2022] POST /identity/v3/users => generated 299 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:10.276214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece64a4-cfdb-4824-814d-978cbdebd2ee admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:10.299327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0df264-b617-4a45-8a21-1d9ebe90b9b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.300052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aece64a4-cfdb-4824-814d-978cbdebd2ee admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.300169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0df264-b617-4a45-8a21-1d9ebe90b9b9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.300389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0df264-b617-4a45-8a21-1d9ebe90b9b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.300673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0df264-b617-4a45-8a21-1d9ebe90b9b9 admin admin] PATH_INFO: `/v3/projects/15c76268fece40ddb334213546ef9700/users/3e03dd87d9244d9ab9275c03994fc360/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.311659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85307abb-a116-4785-96c5-65da73193175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.312913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85307abb-a116-4785-96c5-65da73193175 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.313430 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85307abb-a116-4785-96c5-65da73193175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.314148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85307abb-a116-4785-96c5-65da73193175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.314920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85307abb-a116-4785-96c5-65da73193175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.316643 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85307abb-a116-4785-96c5-65da73193175 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.328517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0df264-b617-4a45-8a21-1d9ebe90b9b9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15c76268fece40ddb334213546ef9700, user_id=3e03dd87d9244d9ab9275c03994fc360, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.331604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0df264-b617-4a45-8a21-1d9ebe90b9b9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.337744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 891/1784] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:10.339594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85307abb-a116-4785-96c5-65da73193175 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:10.357722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 894/1785] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:10 2022] PUT /identity/v3/projects/15c76268fece40ddb334213546ef9700/users/3e03dd87d9244d9ab9275c03994fc360/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:10.364191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85307abb-a116-4785-96c5-65da73193175 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.370417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf97fd-45b8-46c4-bb21-631d785881d6 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.370417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf97fd-45b8-46c4-bb21-631d785881d6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.373897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccf97fd-45b8-46c4-bb21-631d785881d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.416117 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7ccf97fd-45b8-46c4-bb21-631d785881d6 None None] MFA Rules not processed for user `3e03dd87d9244d9ab9275c03994fc360`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:10.417335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7ccf97fd-45b8-46c4-bb21-631d785881d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:10.433307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:10.449333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de448d1-ff28-4266-a7ca-43d4b298a9be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.452739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de448d1-ff28-4266-a7ca-43d4b298a9be admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.453222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de448d1-ff28-4266-a7ca-43d4b298a9be admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.454257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4de448d1-ff28-4266-a7ca-43d4b298a9be admin admin] PATH_INFO: `/v3/projects/41dbcf29c6fc469cb39e3baad6d95629/users/6810a08e92f14b888221f35767b3c929/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.481811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de448d1-ff28-4266-a7ca-43d4b298a9be admin admin] RBAC: Authorizing `identity:create_grant(project_id=41dbcf29c6fc469cb39e3baad6d95629, user_id=6810a08e92f14b888221f35767b3c929, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.486735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4de448d1-ff28-4266-a7ca-43d4b298a9be admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.511233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 892/1786] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:10 2022] PUT /identity/v3/projects/41dbcf29c6fc469cb39e3baad6d95629/users/6810a08e92f14b888221f35767b3c929/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:10.512330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de448d1-ff28-4266-a7ca-43d4b298a9be admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:10.514470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4de448d1-ff28-4266-a7ca-43d4b298a9be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:10.544765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 895/1787] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:10 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:10.545622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf97fd-45b8-46c4-bb21-631d785881d6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:10.558569 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccf97fd-45b8-46c4-bb21-631d785881d6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.593443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:10.625365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4de448d1-ff28-4266-a7ca-43d4b298a9be admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.630138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.632413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.632413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.632413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] PATH_INFO: `/v3/projects/ed7a2a31c0da4b0f8022e3f4d29835d8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.633658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed7a2a31c0da4b0f8022e3f4d29835d8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.635585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.652199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593768a-bc34-4db4-b599-20da6b2aa77e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.652891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593768a-bc34-4db4-b599-20da6b2aa77e admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.653293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593768a-bc34-4db4-b599-20da6b2aa77e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.653450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8593768a-bc34-4db4-b599-20da6b2aa77e admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.654191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593768a-bc34-4db4-b599-20da6b2aa77e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.655830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8593768a-bc34-4db4-b599-20da6b2aa77e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.677290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 896/1788] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:10 2022] GET /identity/v3/roles => generated 1420 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:10.678264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593768a-bc34-4db4-b599-20da6b2aa77e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:10.691908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8593768a-bc34-4db4-b599-20da6b2aa77e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.708609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed7a2a31c0da4b0f8022e3f4d29835d8', 'request_id': 'req-bbd5fc54-1574-4a83-bce8-0eed217544a5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:10.714908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed7a2a31c0da4b0f8022e3f4d29835d8', 'request_id': 'req-bbd5fc54-1574-4a83-bce8-0eed217544a5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:10.723823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed7a2a31c0da4b0f8022e3f4d29835d8', 'request_id': 'req-bbd5fc54-1574-4a83-bce8-0eed217544a5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:10.740046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] The token cache is being invalidate because project ed7a2a31c0da4b0f8022e3f4d29835d8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:10.740609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:10.742238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:10.744807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 893/1789] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:10 2022] DELETE /identity/v3/projects/ed7a2a31c0da4b0f8022e3f4d29835d8 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:10.745676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:10.747119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:10.772881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0822b-f1c0-4eef-b349-73ff01e4b494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.774069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0822b-f1c0-4eef-b349-73ff01e4b494 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.774703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0822b-f1c0-4eef-b349-73ff01e4b494 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.775084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6d0822b-f1c0-4eef-b349-73ff01e4b494 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.775946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0822b-f1c0-4eef-b349-73ff01e4b494 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.778073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6d0822b-f1c0-4eef-b349-73ff01e4b494 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.829242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd5fc54-1574-4a83-bce8-0eed217544a5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.834769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:10.835878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.836170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.836496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] PATH_INFO: `/v3/projects/3e2795852ec64b14b6f52369f005c638` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:10.837878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e2795852ec64b14b6f52369f005c638)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:10.839981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:10.850228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 897/1790] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:10 2022] POST /identity/v3/projects => generated 365 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:10.853203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0822b-f1c0-4eef-b349-73ff01e4b494 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:10.859365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e6d0822b-f1c0-4eef-b349-73ff01e4b494 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:10.925840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:10.931339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e2795852ec64b14b6f52369f005c638', 'request_id': 'req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:10.954002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e2795852ec64b14b6f52369f005c638', 'request_id': 'req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:10.979361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e2795852ec64b14b6f52369f005c638', 'request_id': 'req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:10.985004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6d0822b-f1c0-4eef-b349-73ff01e4b494 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:10.993303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] The token cache is being invalidate because project 3e2795852ec64b14b6f52369f005c638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:10.993698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:10.994448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9c4b4dba-156f-4ee6-8b9a-231e66daa3c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:10.996434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 894/1791] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:10 2022] DELETE /identity/v3/projects/3e2795852ec64b14b6f52369f005c638 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:10.997662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec137c5c-4c19-4a0f-a55d-2103dc793b18 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:10.998028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec137c5c-4c19-4a0f-a55d-2103dc793b18 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:10.998378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec137c5c-4c19-4a0f-a55d-2103dc793b18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.008585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:11.024848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ec137c5c-4c19-4a0f-a55d-2103dc793b18 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:11.026104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ec137c5c-4c19-4a0f-a55d-2103dc793b18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:11.068508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a53067-3688-45c9-b983-641d04ce6a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:11.070395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a53067-3688-45c9-b983-641d04ce6a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.070610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a53067-3688-45c9-b983-641d04ce6a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.070890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a53067-3688-45c9-b983-641d04ce6a88 admin admin] PATH_INFO: `/v3/projects/41dbcf29c6fc469cb39e3baad6d95629/users/6810a08e92f14b888221f35767b3c929/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.074720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a53067-3688-45c9-b983-641d04ce6a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41dbcf29c6fc469cb39e3baad6d95629, user_id=6810a08e92f14b888221f35767b3c929, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:11.078549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a53067-3688-45c9-b983-641d04ce6a88 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:11.097733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 898/1792] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:10 2022] PUT /identity/v3/projects/41dbcf29c6fc469cb39e3baad6d95629/users/6810a08e92f14b888221f35767b3c929/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:11.097733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a53067-3688-45c9-b983-641d04ce6a88 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:11.099562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-51a53067-3688-45c9-b983-641d04ce6a88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:11.109576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 895/1793] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:11.109576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caf919-518c-4e79-83c2-b7691743547a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.109576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caf919-518c-4e79-83c2-b7691743547a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.109576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36caf919-518c-4e79-83c2-b7691743547a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.128488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-36caf919-518c-4e79-83c2-b7691743547a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:11.130358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-36caf919-518c-4e79-83c2-b7691743547a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:11.184745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a53067-3688-45c9-b983-641d04ce6a88 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:11.194871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10fd7a8-0033-457b-8e4f-831029baf2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:11.195739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fd7a8-0033-457b-8e4f-831029baf2b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.196070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fd7a8-0033-457b-8e4f-831029baf2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.196408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10fd7a8-0033-457b-8e4f-831029baf2b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.197733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fd7a8-0033-457b-8e4f-831029baf2b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:11.199274 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10fd7a8-0033-457b-8e4f-831029baf2b2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:11.216261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 896/1794] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:11.218596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7662853-20ae-4196-89a2-2ed90485bf7d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.219029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7662853-20ae-4196-89a2-2ed90485bf7d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.219444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7662853-20ae-4196-89a2-2ed90485bf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.230483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 899/1795] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:11 2022] POST /identity/v3/users => generated 289 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:11.230483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94a6e1-47da-41ce-8708-e83d3aa6d859 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.230483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94a6e1-47da-41ce-8708-e83d3aa6d859 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.230483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e94a6e1-47da-41ce-8708-e83d3aa6d859 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.259574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a7662853-20ae-4196-89a2-2ed90485bf7d None None] MFA Rules not processed for user `6810a08e92f14b888221f35767b3c929`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:11.261169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a7662853-20ae-4196-89a2-2ed90485bf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:11.261480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1e94a6e1-47da-41ce-8708-e83d3aa6d859 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:11.265548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1e94a6e1-47da-41ce-8708-e83d3aa6d859 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:11.292694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 900/1796] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:11.294743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e986e-d400-4c4d-ae96-15099ab811ba None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.295624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e986e-d400-4c4d-ae96-15099ab811ba None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.296060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366e986e-d400-4c4d-ae96-15099ab811ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.332933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-366e986e-d400-4c4d-ae96-15099ab811ba None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:11.334461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-366e986e-d400-4c4d-ae96-15099ab811ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:11.353207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 901/1797] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:11.355377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e986e-d400-4c4d-ae96-15099ab811ba None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:11.369252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366e986e-d400-4c4d-ae96-15099ab811ba None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:11.374695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a67f4-bd56-43d2-a1dd-87b38953d860 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:11.375826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a67f4-bd56-43d2-a1dd-87b38953d860 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.376012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a67f4-bd56-43d2-a1dd-87b38953d860 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.376417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380a67f4-bd56-43d2-a1dd-87b38953d860 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.377268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a67f4-bd56-43d2-a1dd-87b38953d860 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:11.379675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380a67f4-bd56-43d2-a1dd-87b38953d860 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:11.388507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 897/1798] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:11 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:11.389639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7662853-20ae-4196-89a2-2ed90485bf7d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:11.406753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 902/1799] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:11.407750 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a67f4-bd56-43d2-a1dd-87b38953d860 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:11.410733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7662853-20ae-4196-89a2-2ed90485bf7d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:11.418313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409aed8-19b4-437d-a01f-48a1f0b019d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:11.418994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409aed8-19b4-437d-a01f-48a1f0b019d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.419278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409aed8-19b4-437d-a01f-48a1f0b019d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.419552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4409aed8-19b4-437d-a01f-48a1f0b019d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.420283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409aed8-19b4-437d-a01f-48a1f0b019d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:11.420880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409aed8-19b4-437d-a01f-48a1f0b019d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:11.423127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4409aed8-19b4-437d-a01f-48a1f0b019d1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:11.423674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380a67f4-bd56-43d2-a1dd-87b38953d860 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:11.431383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5ef82-a162-4de0-9a9b-c26fd6320792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:11.432184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5ef82-a162-4de0-9a9b-c26fd6320792 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.432571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5ef82-a162-4de0-9a9b-c26fd6320792 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.433190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c5ef82-a162-4de0-9a9b-c26fd6320792 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.433964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5ef82-a162-4de0-9a9b-c26fd6320792 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:11.434877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5ef82-a162-4de0-9a9b-c26fd6320792 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:11.437433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c5ef82-a162-4de0-9a9b-c26fd6320792 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:11.437874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 898/1800] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:11.438801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409aed8-19b4-437d-a01f-48a1f0b019d1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:11.442410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4409aed8-19b4-437d-a01f-48a1f0b019d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:11.456267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 903/1801] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:11.457409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5ef82-a162-4de0-9a9b-c26fd6320792 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:11.473934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:11.475092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c5ef82-a162-4de0-9a9b-c26fd6320792 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:11.480924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96141741-a7c6-415e-a95d-d2f6738d891f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:11.482077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96141741-a7c6-415e-a95d-d2f6738d891f admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.482464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96141741-a7c6-415e-a95d-d2f6738d891f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.482849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96141741-a7c6-415e-a95d-d2f6738d891f admin admin] PATH_INFO: `/v3/projects/08a60f5d910442a29ec4e840e3c09e1e/users/c8033e10b9664bb7be01823840994dfd/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.506131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96141741-a7c6-415e-a95d-d2f6738d891f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a60f5d910442a29ec4e840e3c09e1e, user_id=c8033e10b9664bb7be01823840994dfd, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:11.511087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96141741-a7c6-415e-a95d-d2f6738d891f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:11.524675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:11.536758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4409aed8-19b4-437d-a01f-48a1f0b019d1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:11.537271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 904/1802] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:11 2022] PUT /identity/v3/projects/08a60f5d910442a29ec4e840e3c09e1e/users/c8033e10b9664bb7be01823840994dfd/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:11.538819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96141741-a7c6-415e-a95d-d2f6738d891f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:11.622708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b6be1-cda3-4772-92bf-f53eff81e86c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:11.623377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b6be1-cda3-4772-92bf-f53eff81e86c service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.623664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b6be1-cda3-4772-92bf-f53eff81e86c service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.624081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7b6be1-cda3-4772-92bf-f53eff81e86c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.653356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96141741-a7c6-415e-a95d-d2f6738d891f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:11.661221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adc4ce-a40a-4a6b-81c7-55b21de87ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:11.662235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adc4ce-a40a-4a6b-81c7-55b21de87ff6 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.666225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adc4ce-a40a-4a6b-81c7-55b21de87ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.666735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3adc4ce-a40a-4a6b-81c7-55b21de87ff6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.667439 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3adc4ce-a40a-4a6b-81c7-55b21de87ff6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:11.671481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3adc4ce-a40a-4a6b-81c7-55b21de87ff6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:11.729606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b6be1-cda3-4772-92bf-f53eff81e86c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:11.732340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7b6be1-cda3-4772-92bf-f53eff81e86c service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:11.745386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 899/1803] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:11 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:11.746348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b6be1-cda3-4772-92bf-f53eff81e86c service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:11.760032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7b6be1-cda3-4772-92bf-f53eff81e86c service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:11.763895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 905/1804] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:11 2022] POST /identity/v3/projects => generated 367 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:11.768213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adc4ce-a40a-4a6b-81c7-55b21de87ff6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:11.863930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d7f30-18ea-4f00-a1ee-b0dfe8aebd2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:11.864619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d7f30-18ea-4f00-a1ee-b0dfe8aebd2d admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:11.864905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d7f30-18ea-4f00-a1ee-b0dfe8aebd2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:11.865182 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d7f30-18ea-4f00-a1ee-b0dfe8aebd2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:11.866547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d7f30-18ea-4f00-a1ee-b0dfe8aebd2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:11.868489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d7f30-18ea-4f00-a1ee-b0dfe8aebd2d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:11.915172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 900/1805] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:11 2022] POST /identity/v3/projects => generated 387 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:11.916091 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d7f30-18ea-4f00-a1ee-b0dfe8aebd2d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:11.930425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:11.947626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3adc4ce-a40a-4a6b-81c7-55b21de87ff6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.048119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d7f30-18ea-4f00-a1ee-b0dfe8aebd2d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.067406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd13fd-a4bc-440d-95c5-41aa94b6ebe0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.067828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3493c7-fe67-4003-a330-536c16e33772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.068095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd13fd-a4bc-440d-95c5-41aa94b6ebe0 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.068378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd13fd-a4bc-440d-95c5-41aa94b6ebe0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.068652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fd13fd-a4bc-440d-95c5-41aa94b6ebe0 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.068748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3493c7-fe67-4003-a330-536c16e33772 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.069747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3493c7-fe67-4003-a330-536c16e33772 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.070009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd13fd-a4bc-440d-95c5-41aa94b6ebe0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.070246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a3493c7-fe67-4003-a330-536c16e33772 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.071104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3493c7-fe67-4003-a330-536c16e33772 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.072444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fd13fd-a4bc-440d-95c5-41aa94b6ebe0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.074798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a3493c7-fe67-4003-a330-536c16e33772 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.104150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 906/1806] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:11 2022] GET /identity/v3/roles => generated 1420 bytes in 339 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:12.108175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3493c7-fe67-4003-a330-536c16e33772 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.114384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1a3493c7-fe67-4003-a330-536c16e33772 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.118155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 901/1807] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:11 2022] POST /identity/v3/users => generated 284 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:12.121749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd13fd-a4bc-440d-95c5-41aa94b6ebe0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.128923 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a3493c7-fe67-4003-a330-536c16e33772 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.145093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fd13fd-a4bc-440d-95c5-41aa94b6ebe0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.150669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.151660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.152339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.153336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] PATH_INFO: `/v3/users/6c3ae159bf70463188a39d4fcdc244a0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.155727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c3ae159bf70463188a39d4fcdc244a0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.158121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.162131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d1ac1-2d7e-40b1-afd4-4893b001a5b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.163439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d1ac1-2d7e-40b1-afd4-4893b001a5b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.163957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d1ac1-2d7e-40b1-afd4-4893b001a5b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.164307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66d1ac1-2d7e-40b1-afd4-4893b001a5b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.165083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d1ac1-2d7e-40b1-afd4-4893b001a5b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.168468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a66d1ac1-2d7e-40b1-afd4-4893b001a5b8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.212002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 902/1808] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:12 2022] POST /identity/v3/users => generated 294 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:12.218968 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d1ac1-2d7e-40b1-afd4-4893b001a5b8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.222629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:12.248147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66d1ac1-2d7e-40b1-afd4-4893b001a5b8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.259676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95ebba-4ff1-40f7-9ee6-050f32adedd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.260806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95ebba-4ff1-40f7-9ee6-050f32adedd1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.261350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95ebba-4ff1-40f7-9ee6-050f32adedd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.262272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f95ebba-4ff1-40f7-9ee6-050f32adedd1 admin admin] PATH_INFO: `/v3/projects/08a60f5d910442a29ec4e840e3c09e1e/users/c8033e10b9664bb7be01823840994dfd/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.267167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95ebba-4ff1-40f7-9ee6-050f32adedd1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08a60f5d910442a29ec4e840e3c09e1e, user_id=c8033e10b9664bb7be01823840994dfd, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.270661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f95ebba-4ff1-40f7-9ee6-050f32adedd1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.272576 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c3ae159bf70463188a39d4fcdc244a0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:12.284777 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 903/1809] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:12 2022] PUT /identity/v3/projects/08a60f5d910442a29ec4e840e3c09e1e/users/c8033e10b9664bb7be01823840994dfd/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:12.285791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95ebba-4ff1-40f7-9ee6-050f32adedd1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.314397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c3ae159bf70463188a39d4fcdc244a0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:12.331334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c3ae159bf70463188a39d4fcdc244a0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:12.333350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c3ae159bf70463188a39d4fcdc244a0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:12.334214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c3ae159bf70463188a39d4fcdc244a0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:12.369037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 907/1810] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:12 2022] DELETE /identity/v3/users/6c3ae159bf70463188a39d4fcdc244a0 => generated 0 bytes in 258 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:12.372258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.374091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.453059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f95ebba-4ff1-40f7-9ee6-050f32adedd1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.529353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13c5d54-286b-4f45-a73d-ca72312bf416 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.536532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196b5950-9f4b-4653-a647-9ece1ec7b05b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.538590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b5950-9f4b-4653-a647-9ece1ec7b05b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.539828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b5950-9f4b-4653-a647-9ece1ec7b05b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.540332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196b5950-9f4b-4653-a647-9ece1ec7b05b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.541167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b5950-9f4b-4653-a647-9ece1ec7b05b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.543430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196b5950-9f4b-4653-a647-9ece1ec7b05b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.556519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 908/1811] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:12.558435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d802ce-6937-451e-9aff-1ec7cb6739d1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.558572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d802ce-6937-451e-9aff-1ec7cb6739d1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.558927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d802ce-6937-451e-9aff-1ec7cb6739d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.562441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a50156d-8f35-4d93-8a45-2a96a1c9e250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.563180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50156d-8f35-4d93-8a45-2a96a1c9e250 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.563470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50156d-8f35-4d93-8a45-2a96a1c9e250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.563748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a50156d-8f35-4d93-8a45-2a96a1c9e250 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.564629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50156d-8f35-4d93-8a45-2a96a1c9e250 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.566307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a50156d-8f35-4d93-8a45-2a96a1c9e250 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.585570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 904/1812] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:12.586639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6c6c3-3a5d-461b-a465-afbfd538cc03 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.586953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6c6c3-3a5d-461b-a465-afbfd538cc03 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.587236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d6c6c3-3a5d-461b-a465-afbfd538cc03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.596574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-82d802ce-6937-451e-9aff-1ec7cb6739d1 None None] MFA Rules not processed for user `c8033e10b9664bb7be01823840994dfd`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:12.598145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-82d802ce-6937-451e-9aff-1ec7cb6739d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.611656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-92d6c6c3-3a5d-461b-a465-afbfd538cc03 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:12.614830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-92d6c6c3-3a5d-461b-a465-afbfd538cc03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.627461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 905/1813] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:12.634204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6c6c3-3a5d-461b-a465-afbfd538cc03 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.645021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:12.650176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d6c6c3-3a5d-461b-a465-afbfd538cc03 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.657300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb013c-b79f-4fc0-ac75-af1501637a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.658255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb013c-b79f-4fc0-ac75-af1501637a41 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.659109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb013c-b79f-4fc0-ac75-af1501637a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.659109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2afb013c-b79f-4fc0-ac75-af1501637a41 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.660105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb013c-b79f-4fc0-ac75-af1501637a41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.661861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2afb013c-b79f-4fc0-ac75-af1501637a41 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.678881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 906/1814] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:12.680264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb013c-b79f-4fc0-ac75-af1501637a41 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.682187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2afb013c-b79f-4fc0-ac75-af1501637a41 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.697174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2afb013c-b79f-4fc0-ac75-af1501637a41 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.701545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 909/1815] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:12 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:12.704071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d802ce-6937-451e-9aff-1ec7cb6739d1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.706537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0e0cc-1b9b-44ef-9323-c6f5f7c3054e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.707467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-82d802ce-6937-451e-9aff-1ec7cb6739d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.708649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0e0cc-1b9b-44ef-9323-c6f5f7c3054e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.710283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0e0cc-1b9b-44ef-9323-c6f5f7c3054e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.710582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1d0e0cc-1b9b-44ef-9323-c6f5f7c3054e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.711402 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d0e0cc-1b9b-44ef-9323-c6f5f7c3054e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.714383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1d0e0cc-1b9b-44ef-9323-c6f5f7c3054e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.728547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 907/1816] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:12 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:12.729394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0e0cc-1b9b-44ef-9323-c6f5f7c3054e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.747399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1d0e0cc-1b9b-44ef-9323-c6f5f7c3054e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.753161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1a36f-37e0-4294-9edb-b3cfebcc9862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.754201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1a36f-37e0-4294-9edb-b3cfebcc9862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.754318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1a36f-37e0-4294-9edb-b3cfebcc9862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.754680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c1a36f-37e0-4294-9edb-b3cfebcc9862 admin admin] PATH_INFO: `/v3/projects/4b59e674e34f4f9cbc1619e2f4c0ad9e/users/9275e136c22c4a5da114cc3ddb02df7a/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:12.759462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:12.786936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1a36f-37e0-4294-9edb-b3cfebcc9862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b59e674e34f4f9cbc1619e2f4c0ad9e, user_id=9275e136c22c4a5da114cc3ddb02df7a, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:12.792863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c1a36f-37e0-4294-9edb-b3cfebcc9862 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:12.808877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 908/1817] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:12 2022] PUT /identity/v3/projects/4b59e674e34f4f9cbc1619e2f4c0ad9e/users/9275e136c22c4a5da114cc3ddb02df7a/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:12.809787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1a36f-37e0-4294-9edb-b3cfebcc9862 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:12.811592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a4c1a36f-37e0-4294-9edb-b3cfebcc9862 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:12.814664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d802ce-6937-451e-9aff-1ec7cb6739d1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:12.939671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3fb1af-210d-46bf-a66e-d8121bdd0a16 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:12.940452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3fb1af-210d-46bf-a66e-d8121bdd0a16 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:12.940814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3fb1af-210d-46bf-a66e-d8121bdd0a16 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:12.941166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3fb1af-210d-46bf-a66e-d8121bdd0a16 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.002121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c1a36f-37e0-4294-9edb-b3cfebcc9862 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:13.010392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab89ab2e-f417-411d-9bea-cd415e9d4fbb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:13.011436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89ab2e-f417-411d-9bea-cd415e9d4fbb service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:13.011878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89ab2e-f417-411d-9bea-cd415e9d4fbb service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:13.012302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab89ab2e-f417-411d-9bea-cd415e9d4fbb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.097187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3fb1af-210d-46bf-a66e-d8121bdd0a16 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:13.099428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89ab2e-f417-411d-9bea-cd415e9d4fbb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:13.100945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3fb1af-210d-46bf-a66e-d8121bdd0a16 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:13.102653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab89ab2e-f417-411d-9bea-cd415e9d4fbb service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:13.114450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 910/1818] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:13.117007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3fb1af-210d-46bf-a66e-d8121bdd0a16 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:13.125107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 909/1819] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:12 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:13.133114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72fef-f248-470d-b95c-dedd5dbee696 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:13.133688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72fef-f248-470d-b95c-dedd5dbee696 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:13.134164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d72fef-f248-470d-b95c-dedd5dbee696 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.140932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3fb1af-210d-46bf-a66e-d8121bdd0a16 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:13.148092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e97c284-2cdf-4f55-b41c-ded5ba17fc90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:13.150024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97c284-2cdf-4f55-b41c-ded5ba17fc90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:13.150329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97c284-2cdf-4f55-b41c-ded5ba17fc90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:13.150614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e97c284-2cdf-4f55-b41c-ded5ba17fc90 admin admin] PATH_INFO: `/v3/projects/d140ff225d104aa59e4262febe1a6b8e/users/d9bece2e8ca849db8fa9bfe853f57209/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.179038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97c284-2cdf-4f55-b41c-ded5ba17fc90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d140ff225d104aa59e4262febe1a6b8e, user_id=d9bece2e8ca849db8fa9bfe853f57209, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:13.182361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e97c284-2cdf-4f55-b41c-ded5ba17fc90 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:13.189795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-84d72fef-f248-470d-b95c-dedd5dbee696 None None] MFA Rules not processed for user `9275e136c22c4a5da114cc3ddb02df7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:13.192106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-84d72fef-f248-470d-b95c-dedd5dbee696 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:13.220412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 911/1820] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:13 2022] PUT /identity/v3/projects/d140ff225d104aa59e4262febe1a6b8e/users/d9bece2e8ca849db8fa9bfe853f57209/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:13.231557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4900-d606-4ddb-b483-1d630190b5b3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:13.232391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4900-d606-4ddb-b483-1d630190b5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:13.232863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873b4900-d606-4ddb-b483-1d630190b5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.297377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-873b4900-d606-4ddb-b483-1d630190b5b3 None None] MFA Rules not processed for user `d9bece2e8ca849db8fa9bfe853f57209`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:13.299614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-873b4900-d606-4ddb-b483-1d630190b5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:13.343786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:13.354656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 910/1821] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:13 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:13.360825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d72fef-f248-470d-b95c-dedd5dbee696 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:13.374964 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d72fef-f248-470d-b95c-dedd5dbee696 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:13.466446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:13.541114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 912/1822] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:13 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 311 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:13.543164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b4900-d606-4ddb-b483-1d630190b5b3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:13.568105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeba87-a927-4d65-aeb2-d2cbdf024c2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:13.568793 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeba87-a927-4d65-aeb2-d2cbdf024c2e admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:13.569116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeba87-a927-4d65-aeb2-d2cbdf024c2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:13.571360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3baeba87-a927-4d65-aeb2-d2cbdf024c2e admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.573272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baeba87-a927-4d65-aeb2-d2cbdf024c2e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:13.575154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3baeba87-a927-4d65-aeb2-d2cbdf024c2e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:13.651617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 911/1823] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:13 2022] POST /identity/v3/projects => generated 387 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:13.654682 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeba87-a927-4d65-aeb2-d2cbdf024c2e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:13.725656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873b4900-d606-4ddb-b483-1d630190b5b3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:13.823776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3baeba87-a927-4d65-aeb2-d2cbdf024c2e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:13.833177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fdb34-4eff-4a7a-b66f-68e1a45d64b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:13.833908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fdb34-4eff-4a7a-b66f-68e1a45d64b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:13.834193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fdb34-4eff-4a7a-b66f-68e1a45d64b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:13.834564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002fdb34-4eff-4a7a-b66f-68e1a45d64b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.836318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fdb34-4eff-4a7a-b66f-68e1a45d64b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:13.838423 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-002fdb34-4eff-4a7a-b66f-68e1a45d64b1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:13.863488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5658db2-8568-4932-8125-05b67b8dfe37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:13.864187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5658db2-8568-4932-8125-05b67b8dfe37 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:13.864475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5658db2-8568-4932-8125-05b67b8dfe37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:13.864751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5658db2-8568-4932-8125-05b67b8dfe37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:13.867737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5658db2-8568-4932-8125-05b67b8dfe37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:13.875312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5658db2-8568-4932-8125-05b67b8dfe37 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:13.931618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 912/1824] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:13 2022] POST /identity/v3/users => generated 300 bytes in 279 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:13.932671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fdb34-4eff-4a7a-b66f-68e1a45d64b1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:13.941264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-002fdb34-4eff-4a7a-b66f-68e1a45d64b1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:13.946164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 913/1825] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:13 2022] POST /identity/v3/projects => generated 367 bytes in 404 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:13.951282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5658db2-8568-4932-8125-05b67b8dfe37 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:13.962891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-002fdb34-4eff-4a7a-b66f-68e1a45d64b1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.044268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.045547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.046054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.046492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] PATH_INFO: `/v3/projects/6ecad7bc6ee1494dbfa7c9c5fa47c97b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.048523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6ecad7bc6ee1494dbfa7c9c5fa47c97b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.050673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.051903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5658db2-8568-4932-8125-05b67b8dfe37 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.057398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc406cf5-f2a9-4602-8a0a-42c1ab298b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.058318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc406cf5-f2a9-4602-8a0a-42c1ab298b58 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.058692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc406cf5-f2a9-4602-8a0a-42c1ab298b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.059033 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc406cf5-f2a9-4602-8a0a-42c1ab298b58 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.059947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc406cf5-f2a9-4602-8a0a-42c1ab298b58 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.061677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc406cf5-f2a9-4602-8a0a-42c1ab298b58 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.071656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:14.073607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 914/1826] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:13 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:14.074573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc406cf5-f2a9-4602-8a0a-42c1ab298b58 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.086239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc406cf5-f2a9-4602-8a0a-42c1ab298b58 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.095012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9154aa2-dcee-498c-b4e7-f6ee6ee30b66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.096192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9154aa2-dcee-498c-b4e7-f6ee6ee30b66 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.096770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9154aa2-dcee-498c-b4e7-f6ee6ee30b66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.097634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9154aa2-dcee-498c-b4e7-f6ee6ee30b66 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.099186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9154aa2-dcee-498c-b4e7-f6ee6ee30b66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.104417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9154aa2-dcee-498c-b4e7-f6ee6ee30b66 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.116755 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6ecad7bc6ee1494dbfa7c9c5fa47c97b', 'request_id': 'req-473e91d2-d322-4726-b1d5-89042e92af95'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.127228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6ecad7bc6ee1494dbfa7c9c5fa47c97b', 'request_id': 'req-473e91d2-d322-4726-b1d5-89042e92af95'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.133170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6ecad7bc6ee1494dbfa7c9c5fa47c97b', 'request_id': 'req-473e91d2-d322-4726-b1d5-89042e92af95'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.137829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 915/1827] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:14 2022] POST /identity/v3/users => generated 290 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:14.138988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9154aa2-dcee-498c-b4e7-f6ee6ee30b66 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.140746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e9154aa2-dcee-498c-b4e7-f6ee6ee30b66 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.148422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] The token cache is being invalidate because project 6ecad7bc6ee1494dbfa7c9c5fa47c97b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:14.148797 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.149741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.151845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:14.152366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 913/1828] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:13 2022] DELETE /identity/v3/projects/6ecad7bc6ee1494dbfa7c9c5fa47c97b => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:14.153773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.155773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.157643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9154aa2-dcee-498c-b4e7-f6ee6ee30b66 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.227205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-473e91d2-d322-4726-b1d5-89042e92af95 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.233009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52cbdd-77ba-42c6-a548-962d3e865e09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.233829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52cbdd-77ba-42c6-a548-962d3e865e09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.236812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52cbdd-77ba-42c6-a548-962d3e865e09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.236812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b52cbdd-77ba-42c6-a548-962d3e865e09 admin admin] PATH_INFO: `/v3/projects/03d63ab890ae488b8d0673b9dcb458c1/users/3cd07cb936424e578bed7695432434ea/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.245387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.246412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.246761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.247138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] PATH_INFO: `/v3/users/3e03dd87d9244d9ab9275c03994fc360` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.248649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3e03dd87d9244d9ab9275c03994fc360)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.250839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.251174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52cbdd-77ba-42c6-a548-962d3e865e09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d63ab890ae488b8d0673b9dcb458c1, user_id=3cd07cb936424e578bed7695432434ea, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.253936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b52cbdd-77ba-42c6-a548-962d3e865e09 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.271628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 914/1829] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:14 2022] PUT /identity/v3/projects/03d63ab890ae488b8d0673b9dcb458c1/users/3cd07cb936424e578bed7695432434ea/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:14.273203 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52cbdd-77ba-42c6-a548-962d3e865e09 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.276885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8b52cbdd-77ba-42c6-a548-962d3e865e09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.327605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3e03dd87d9244d9ab9275c03994fc360'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.344202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3e03dd87d9244d9ab9275c03994fc360'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.352471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3e03dd87d9244d9ab9275c03994fc360'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.353384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3e03dd87d9244d9ab9275c03994fc360'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.354280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3e03dd87d9244d9ab9275c03994fc360'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.368552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 916/1830] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:14 2022] DELETE /identity/v3/users/3e03dd87d9244d9ab9275c03994fc360 => generated 0 bytes in 230 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:14.369370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.370742 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b52cbdd-77ba-42c6-a548-962d3e865e09 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.371564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.402553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:14.432612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891a31-0cf2-44bd-a8c4-1e21d785e616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.433308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891a31-0cf2-44bd-a8c4-1e21d785e616 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.433638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891a31-0cf2-44bd-a8c4-1e21d785e616 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.433916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94891a31-0cf2-44bd-a8c4-1e21d785e616 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.434647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94891a31-0cf2-44bd-a8c4-1e21d785e616 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.436264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94891a31-0cf2-44bd-a8c4-1e21d785e616 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.441885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca866d0-f693-4937-ae03-fd7741196dc2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.446444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 915/1831] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:14.446661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.447254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891a31-0cf2-44bd-a8c4-1e21d785e616 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.447544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.447962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.448247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] PATH_INFO: `/v3/users/6810a08e92f14b888221f35767b3c929` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.449285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-94891a31-0cf2-44bd-a8c4-1e21d785e616 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.449602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] RBAC: Authorizing `identity:delete_user(user_id=6810a08e92f14b888221f35767b3c929)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.451593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.462003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94891a31-0cf2-44bd-a8c4-1e21d785e616 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.466693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a673eaec-95ee-47ff-9da4-54c69310f1b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.467747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673eaec-95ee-47ff-9da4-54c69310f1b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.468031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673eaec-95ee-47ff-9da4-54c69310f1b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.468313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a673eaec-95ee-47ff-9da4-54c69310f1b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.469036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673eaec-95ee-47ff-9da4-54c69310f1b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.470679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a673eaec-95ee-47ff-9da4-54c69310f1b6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.481504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 916/1832] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:14.482598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:14.482854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6d749-cefe-4a96-b1b8-1c241733b070 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.483153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6d749-cefe-4a96-b1b8-1c241733b070 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.483431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a6d749-cefe-4a96-b1b8-1c241733b070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.503721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a4a6d749-cefe-4a96-b1b8-1c241733b070 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:14.504934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a4a6d749-cefe-4a96-b1b8-1c241733b070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.512744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 917/1833] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:14.513581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6d749-cefe-4a96-b1b8-1c241733b070 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.514873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a4a6d749-cefe-4a96-b1b8-1c241733b070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.515760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6810a08e92f14b888221f35767b3c929'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.527027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a6d749-cefe-4a96-b1b8-1c241733b070 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.531484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6810a08e92f14b888221f35767b3c929'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.531939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a115dac-efcd-4dc8-849c-bb45b9a22fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.532830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a115dac-efcd-4dc8-849c-bb45b9a22fa8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.533115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a115dac-efcd-4dc8-849c-bb45b9a22fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.533396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a115dac-efcd-4dc8-849c-bb45b9a22fa8 admin admin] PATH_INFO: `/v3/projects/71eba902a494470cad8a0cc764e344e8/users/0ab7eb07874c413dbebe7e51ce042a1f/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.540471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6810a08e92f14b888221f35767b3c929'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.541381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6810a08e92f14b888221f35767b3c929'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.542194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6810a08e92f14b888221f35767b3c929'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:14.554703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a115dac-efcd-4dc8-849c-bb45b9a22fa8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=71eba902a494470cad8a0cc764e344e8, user_id=0ab7eb07874c413dbebe7e51ce042a1f, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.557462 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a115dac-efcd-4dc8-849c-bb45b9a22fa8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.564051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 917/1834] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:14 2022] DELETE /identity/v3/users/6810a08e92f14b888221f35767b3c929 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:14.566376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.570975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.579052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 918/1835] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:14 2022] PUT /identity/v3/projects/71eba902a494470cad8a0cc764e344e8/users/0ab7eb07874c413dbebe7e51ce042a1f/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:14.580082 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a115dac-efcd-4dc8-849c-bb45b9a22fa8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.584624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5a115dac-efcd-4dc8-849c-bb45b9a22fa8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.665627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a115dac-efcd-4dc8-849c-bb45b9a22fa8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.669899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582904ca-67e8-46bf-aa2d-32631a3878bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.670963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582904ca-67e8-46bf-aa2d-32631a3878bc service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.671339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582904ca-67e8-46bf-aa2d-32631a3878bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.671755 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582904ca-67e8-46bf-aa2d-32631a3878bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.673902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-582904ca-67e8-46bf-aa2d-32631a3878bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.682044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92465a2f-4a77-4dd1-b08d-4b93a47f245a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.765999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57915ee-6d3c-46ac-a985-447d949125f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.767498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57915ee-6d3c-46ac-a985-447d949125f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.767498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57915ee-6d3c-46ac-a985-447d949125f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.767747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a57915ee-6d3c-46ac-a985-447d949125f6 admin admin] PATH_INFO: `/v3/projects/03d63ab890ae488b8d0673b9dcb458c1/users/3cd07cb936424e578bed7695432434ea/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.769950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57915ee-6d3c-46ac-a985-447d949125f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=03d63ab890ae488b8d0673b9dcb458c1, user_id=3cd07cb936424e578bed7695432434ea, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.773710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a57915ee-6d3c-46ac-a985-447d949125f6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.775356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582904ca-67e8-46bf-aa2d-32631a3878bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.777741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-582904ca-67e8-46bf-aa2d-32631a3878bc service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.787034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 918/1836] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:14 2022] PUT /identity/v3/projects/03d63ab890ae488b8d0673b9dcb458c1/users/3cd07cb936424e578bed7695432434ea/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:14.794044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57915ee-6d3c-46ac-a985-447d949125f6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.796483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a57915ee-6d3c-46ac-a985-447d949125f6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.797710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 919/1837] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:14.799205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c479eb-fbfa-4a68-9c98-3e34f3319c90 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.799502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c479eb-fbfa-4a68-9c98-3e34f3319c90 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.799782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c479eb-fbfa-4a68-9c98-3e34f3319c90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.833308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f0c479eb-fbfa-4a68-9c98-3e34f3319c90 None None] MFA Rules not processed for user `3cd07cb936424e578bed7695432434ea`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:14.835452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f0c479eb-fbfa-4a68-9c98-3e34f3319c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.886967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:14.910917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=6,keystone:SELECT=92,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:14.936808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a57915ee-6d3c-46ac-a985-447d949125f6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.941989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe8bfd-45c2-432e-a06d-b500f4411fc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.943195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe8bfd-45c2-432e-a06d-b500f4411fc5 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.943327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe8bfd-45c2-432e-a06d-b500f4411fc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.943533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5fe8bfd-45c2-432e-a06d-b500f4411fc5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.944545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe8bfd-45c2-432e-a06d-b500f4411fc5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.946403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5fe8bfd-45c2-432e-a06d-b500f4411fc5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:14.947971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 920/1838] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:14 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:14.955254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c479eb-fbfa-4a68-9c98-3e34f3319c90 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.958412 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f0c479eb-fbfa-4a68-9c98-3e34f3319c90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:14.965821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 919/1839] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:14 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:14.970692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe8bfd-45c2-432e-a06d-b500f4411fc5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:14.984348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5fe8bfd-45c2-432e-a06d-b500f4411fc5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:14.990071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40ff931-4c16-401a-baa3-6e7d55e9b17d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:14.992015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ff931-4c16-401a-baa3-6e7d55e9b17d admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:14.992376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ff931-4c16-401a-baa3-6e7d55e9b17d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:14.992727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40ff931-4c16-401a-baa3-6e7d55e9b17d admin admin] PATH_INFO: `/v3/projects/71eba902a494470cad8a0cc764e344e8/users/0ab7eb07874c413dbebe7e51ce042a1f/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:14.994947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ff931-4c16-401a-baa3-6e7d55e9b17d admin admin] RBAC: Authorizing `identity:create_grant(project_id=71eba902a494470cad8a0cc764e344e8, user_id=0ab7eb07874c413dbebe7e51ce042a1f, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:14.998199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40ff931-4c16-401a-baa3-6e7d55e9b17d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:15.017632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 920/1840] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:14 2022] PUT /identity/v3/projects/71eba902a494470cad8a0cc764e344e8/users/0ab7eb07874c413dbebe7e51ce042a1f/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:15.026347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf66d-a962-498c-80ab-1b8edfbf2e62 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:15.026786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf66d-a962-498c-80ab-1b8edfbf2e62 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:15.027078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96dcf66d-a962-498c-80ab-1b8edfbf2e62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:15.049696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c479eb-fbfa-4a68-9c98-3e34f3319c90 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:15.070921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-96dcf66d-a962-498c-80ab-1b8edfbf2e62 None None] MFA Rules not processed for user `0ab7eb07874c413dbebe7e51ce042a1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:15.072220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-96dcf66d-a962-498c-80ab-1b8edfbf2e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:15.168383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c532e6-a7cb-47c9-8069-9807603ebcd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:15.169058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c532e6-a7cb-47c9-8069-9807603ebcd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:15.169484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c532e6-a7cb-47c9-8069-9807603ebcd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:15.169649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c532e6-a7cb-47c9-8069-9807603ebcd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:15.189161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 921/1841] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:15 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:15.190197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcf66d-a962-498c-80ab-1b8edfbf2e62 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:15.192069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-96dcf66d-a962-498c-80ab-1b8edfbf2e62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:15.283680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c532e6-a7cb-47c9-8069-9807603ebcd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:15.285235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c532e6-a7cb-47c9-8069-9807603ebcd3 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:15.291992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96dcf66d-a962-498c-80ab-1b8edfbf2e62 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:15.298229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:15.300489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:15.303078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:15.304242 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] PATH_INFO: `/v3/projects/41dbcf29c6fc469cb39e3baad6d95629` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:15.305896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] RBAC: Authorizing `identity:delete_project(project_id=41dbcf29c6fc469cb39e3baad6d95629)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:15.310316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:15.332103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 921/1842] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:14 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:15.333371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c532e6-a7cb-47c9-8069-9807603ebcd3 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:15.338810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-40c532e6-a7cb-47c9-8069-9807603ebcd3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:15.370167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c532e6-a7cb-47c9-8069-9807603ebcd3 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:15.378475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec8e6e-546f-46f4-b3e1-d480dadb8393 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:15.379161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec8e6e-546f-46f4-b3e1-d480dadb8393 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:15.379428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec8e6e-546f-46f4-b3e1-d480dadb8393 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:15.379698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5ec8e6e-546f-46f4-b3e1-d480dadb8393 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:15.393607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec8e6e-546f-46f4-b3e1-d480dadb8393 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:15.395064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5ec8e6e-546f-46f4-b3e1-d480dadb8393 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:15.404611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41dbcf29c6fc469cb39e3baad6d95629', 'request_id': 'req-401c0036-3760-4e2c-8336-86d7dd5515db'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.406018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 922/1843] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:15 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:15.406952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec8e6e-546f-46f4-b3e1-d480dadb8393 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:15.416115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41dbcf29c6fc469cb39e3baad6d95629', 'request_id': 'req-401c0036-3760-4e2c-8336-86d7dd5515db'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.418541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f5ec8e6e-546f-46f4-b3e1-d480dadb8393 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:15.420212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:15.432497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41dbcf29c6fc469cb39e3baad6d95629', 'request_id': 'req-401c0036-3760-4e2c-8336-86d7dd5515db'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.445173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5ec8e6e-546f-46f4-b3e1-d480dadb8393 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:15.466295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:15.471449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] The token cache is being invalidate because project 41dbcf29c6fc469cb39e3baad6d95629 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:15.472522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.487409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.500729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 922/1844] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:15 2022] DELETE /identity/v3/projects/41dbcf29c6fc469cb39e3baad6d95629 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:15.507799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:15.512346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:15.540761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:15.542661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:15.543157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:15.543617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] PATH_INFO: `/v3/projects/15c76268fece40ddb334213546ef9700` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:15.545799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=15c76268fece40ddb334213546ef9700)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:15.549053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:15.626008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15c76268fece40ddb334213546ef9700', 'request_id': 'req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.632543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15c76268fece40ddb334213546ef9700', 'request_id': 'req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.639990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401c0036-3760-4e2c-8336-86d7dd5515db admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:15.645071 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15c76268fece40ddb334213546ef9700', 'request_id': 'req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.663211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] The token cache is being invalidate because project 15c76268fece40ddb334213546ef9700 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:15.663657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.664349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8a307c0c-2d58-4ed5-a3a7-5cbf994e90c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.666387 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 923/1845] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:15 2022] DELETE /identity/v3/projects/15c76268fece40ddb334213546ef9700 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:15.667518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a50f91-c82c-413b-bb30-c5840cbaa682 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:15.667807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a50f91-c82c-413b-bb30-c5840cbaa682 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:15.668081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a50f91-c82c-413b-bb30-c5840cbaa682 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:15.689579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-11a50f91-c82c-413b-bb30-c5840cbaa682 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:15.690953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-11a50f91-c82c-413b-bb30-c5840cbaa682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:15.711042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:15.712015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:15.712358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:15.712699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] PATH_INFO: `/v3/users/9275e136c22c4a5da114cc3ddb02df7a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:15.714003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9275e136c22c4a5da114cc3ddb02df7a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:15.716008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:15.767412 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 924/1846] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:15.768593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a50f91-c82c-413b-bb30-c5840cbaa682 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:15.769903 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-11a50f91-c82c-413b-bb30-c5840cbaa682 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:15.774247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9275e136c22c4a5da114cc3ddb02df7a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.788470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9275e136c22c4a5da114cc3ddb02df7a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.799260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9275e136c22c4a5da114cc3ddb02df7a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.800397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9275e136c22c4a5da114cc3ddb02df7a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.801188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9275e136c22c4a5da114cc3ddb02df7a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.816285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 923/1847] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:15 2022] DELETE /identity/v3/users/9275e136c22c4a5da114cc3ddb02df7a => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:15.817337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:15.821190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:15.829059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:15.834248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:15.837171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a50f91-c82c-413b-bb30-c5840cbaa682 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:15.900100 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af434e2b-ed9e-41cf-b508-8d0857277231 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:15.904862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:15.905778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:15.906060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:15.906333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] PATH_INFO: `/v3/users/c8033e10b9664bb7be01823840994dfd` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:15.907610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c8033e10b9664bb7be01823840994dfd)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:15.908236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfb8543-452e-4065-9971-54a6cc1a627c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:15.908901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb8543-452e-4065-9971-54a6cc1a627c service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:15.909314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb8543-452e-4065-9971-54a6cc1a627c service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:15.909537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:15.909652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfb8543-452e-4065-9971-54a6cc1a627c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:15.911606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2dfb8543-452e-4065-9971-54a6cc1a627c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:15.966687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c8033e10b9664bb7be01823840994dfd'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.981057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c8033e10b9664bb7be01823840994dfd'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.989945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c8033e10b9664bb7be01823840994dfd'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.990911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c8033e10b9664bb7be01823840994dfd'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.991807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3aacd3db-75ef-4429-8aa8-fdb30fcd8c63 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c8033e10b9664bb7be01823840994dfd'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:15.993031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb8543-452e-4065-9971-54a6cc1a627c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:15.994226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfb8543-452e-4065-9971-54a6cc1a627c service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.006119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 925/1848] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:15 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:16.008602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a784f6-cd4d-4dfe-80a9-4f83ef376e4b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.009045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a784f6-cd4d-4dfe-80a9-4f83ef376e4b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.009466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a784f6-cd4d-4dfe-80a9-4f83ef376e4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.019457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 924/1849] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:15 2022] DELETE /identity/v3/users/c8033e10b9664bb7be01823840994dfd => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:16.021825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f1860-2323-4225-af50-a809e327fc8e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.022755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f1860-2323-4225-af50-a809e327fc8e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.023045 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6f1860-2323-4225-af50-a809e327fc8e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.034186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e1a784f6-cd4d-4dfe-80a9-4f83ef376e4b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:16.035810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e1a784f6-cd4d-4dfe-80a9-4f83ef376e4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:16.054665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-fc6f1860-2323-4225-af50-a809e327fc8e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:16.056426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fc6f1860-2323-4225-af50-a809e327fc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:16.151205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 926/1850] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:16.157914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a784f6-cd4d-4dfe-80a9-4f83ef376e4b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.180750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a784f6-cd4d-4dfe-80a9-4f83ef376e4b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.192281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3fd53-3f0f-429b-9515-6e3d5f6fbfe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:16.193697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3fd53-3f0f-429b-9515-6e3d5f6fbfe8 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.198327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3fd53-3f0f-429b-9515-6e3d5f6fbfe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.198827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e3fd53-3f0f-429b-9515-6e3d5f6fbfe8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.199562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 925/1851] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:16.204401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3fd53-3f0f-429b-9515-6e3d5f6fbfe8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:16.204535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3fd53-3f0f-429b-9515-6e3d5f6fbfe8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:16.207000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e3fd53-3f0f-429b-9515-6e3d5f6fbfe8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.209428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f1860-2323-4225-af50-a809e327fc8e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.210868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fc6f1860-2323-4225-af50-a809e327fc8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:16.232242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 927/1852] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:24:16.236511 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3fd53-3f0f-429b-9515-6e3d5f6fbfe8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.268766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e3fd53-3f0f-429b-9515-6e3d5f6fbfe8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.270163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:16.279685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6fad2-01f4-4a87-afef-94ff379edb36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:16.280917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6fad2-01f4-4a87-afef-94ff379edb36 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.280917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6fad2-01f4-4a87-afef-94ff379edb36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.281151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f6fad2-01f4-4a87-afef-94ff379edb36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.281844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6fad2-01f4-4a87-afef-94ff379edb36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:16.283951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f6fad2-01f4-4a87-afef-94ff379edb36 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.324651 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 928/1853] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:16 2022] POST /identity/v3/projects => generated 377 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:16.331050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6fad2-01f4-4a87-afef-94ff379edb36 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.356137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc6f1860-2323-4225-af50-a809e327fc8e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.394655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:16.462639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728c4c1-becd-480a-bcc3-f76d52fc759a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:16.464029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728c4c1-becd-480a-bcc3-f76d52fc759a service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.464318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728c4c1-becd-480a-bcc3-f76d52fc759a service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.464623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2728c4c1-becd-480a-bcc3-f76d52fc759a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.540359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f6fad2-01f4-4a87-afef-94ff379edb36 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.552186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26111cea-0ab1-4f24-a0d4-ed063ced6787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:16.553381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26111cea-0ab1-4f24-a0d4-ed063ced6787 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.554053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26111cea-0ab1-4f24-a0d4-ed063ced6787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.554597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26111cea-0ab1-4f24-a0d4-ed063ced6787 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.556262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26111cea-0ab1-4f24-a0d4-ed063ced6787 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:16.559924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26111cea-0ab1-4f24-a0d4-ed063ced6787 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.608508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728c4c1-becd-480a-bcc3-f76d52fc759a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:16.610513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2728c4c1-becd-480a-bcc3-f76d52fc759a service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.626625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 929/1854] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:16 2022] POST /identity/v3/users => generated 289 bytes in 294 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:16.626625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26111cea-0ab1-4f24-a0d4-ed063ced6787 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.629876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-26111cea-0ab1-4f24-a0d4-ed063ced6787 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:16.651596 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 926/1855] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:16.652532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728c4c1-becd-480a-bcc3-f76d52fc759a service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.664753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2728c4c1-becd-480a-bcc3-f76d52fc759a service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.673002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f90a99-be7c-45f8-9cd4-9cd5deeea261 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:16.673942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f90a99-be7c-45f8-9cd4-9cd5deeea261 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.674295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f90a99-be7c-45f8-9cd4-9cd5deeea261 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.674644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86f90a99-be7c-45f8-9cd4-9cd5deeea261 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.675446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f90a99-be7c-45f8-9cd4-9cd5deeea261 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:16.677156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86f90a99-be7c-45f8-9cd4-9cd5deeea261 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.680751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26111cea-0ab1-4f24-a0d4-ed063ced6787 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.698772 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 927/1856] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:16.700566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:16.701676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f90a99-be7c-45f8-9cd4-9cd5deeea261 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.702779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.703538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.705878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] PATH_INFO: `/v3/projects/08a60f5d910442a29ec4e840e3c09e1e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.708820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08a60f5d910442a29ec4e840e3c09e1e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:16.713203 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.729667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86f90a99-be7c-45f8-9cd4-9cd5deeea261 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.750509 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a2f8-f44c-43c2-9e2a-f9e09a30b205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:16.751459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a2f8-f44c-43c2-9e2a-f9e09a30b205 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:16.751752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a2f8-f44c-43c2-9e2a-f9e09a30b205 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:16.752083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e6a2f8-f44c-43c2-9e2a-f9e09a30b205 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:16.752855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a2f8-f44c-43c2-9e2a-f9e09a30b205 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:16.754714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e6a2f8-f44c-43c2-9e2a-f9e09a30b205 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:16.796616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 928/1857] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:16 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:16.798508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a2f8-f44c-43c2-9e2a-f9e09a30b205 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:16.841634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e6a2f8-f44c-43c2-9e2a-f9e09a30b205 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:16.900707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08a60f5d910442a29ec4e840e3c09e1e', 'request_id': 'req-f4972b2b-8596-48a8-856f-14c1a2c17723'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.935594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08a60f5d910442a29ec4e840e3c09e1e', 'request_id': 'req-f4972b2b-8596-48a8-856f-14c1a2c17723'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.951677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08a60f5d910442a29ec4e840e3c09e1e', 'request_id': 'req-f4972b2b-8596-48a8-856f-14c1a2c17723'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.984883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] The token cache is being invalidate because project 08a60f5d910442a29ec4e840e3c09e1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:16.985387 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:16.998287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f4972b2b-8596-48a8-856f-14c1a2c17723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.001161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 930/1858] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:16 2022] DELETE /identity/v3/projects/08a60f5d910442a29ec4e840e3c09e1e => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:17.009169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468033f-6d6c-464a-840f-b5d2621578f4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.009705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468033f-6d6c-464a-840f-b5d2621578f4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.010198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2468033f-6d6c-464a-840f-b5d2621578f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.064771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9f12d-513f-49c5-a37d-14407ae21e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.066398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9f12d-513f-49c5-a37d-14407ae21e4c admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.066906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9f12d-513f-49c5-a37d-14407ae21e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.067519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede9f12d-513f-49c5-a37d-14407ae21e4c admin admin] PATH_INFO: `/v3/projects/5dd491b6a7da44d68f496d142fce9267/users/7923d8bc001b49e28518dad0bafaa002/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.070369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2468033f-6d6c-464a-840f-b5d2621578f4 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:17.076026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2468033f-6d6c-464a-840f-b5d2621578f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:17.097879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 931/1859] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:17.110613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e496d7-44db-47ba-87b3-4d32b30a1b25 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.111145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e496d7-44db-47ba-87b3-4d32b30a1b25 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.111578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e496d7-44db-47ba-87b3-4d32b30a1b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.115099 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9f12d-513f-49c5-a37d-14407ae21e4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5dd491b6a7da44d68f496d142fce9267, user_id=7923d8bc001b49e28518dad0bafaa002, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.121050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede9f12d-513f-49c5-a37d-14407ae21e4c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.136597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:17.164683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 929/1860] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:16 2022] PUT /identity/v3/projects/5dd491b6a7da44d68f496d142fce9267/users/7923d8bc001b49e28518dad0bafaa002/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:17.165809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9f12d-513f-49c5-a37d-14407ae21e4c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.167398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-38e496d7-44db-47ba-87b3-4d32b30a1b25 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:17.167860 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ede9f12d-513f-49c5-a37d-14407ae21e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:17.171018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-38e496d7-44db-47ba-87b3-4d32b30a1b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:17.263608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede9f12d-513f-49c5-a37d-14407ae21e4c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.274209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.279407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 932/1861] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:17.279751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.279751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.279751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] PATH_INFO: `/v3/projects/4b59e674e34f4f9cbc1619e2f4c0ad9e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.279999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0b6cf-506a-4721-8259-5eb5b944bb50 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.280473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b59e674e34f4f9cbc1619e2f4c0ad9e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.281705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0b6cf-506a-4721-8259-5eb5b944bb50 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.282170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d0b6cf-506a-4721-8259-5eb5b944bb50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.283992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.322916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-50d0b6cf-506a-4721-8259-5eb5b944bb50 None None] MFA Rules not processed for user `7923d8bc001b49e28518dad0bafaa002`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:17.324917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-50d0b6cf-506a-4721-8259-5eb5b944bb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:17.343885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b59e674e34f4f9cbc1619e2f4c0ad9e', 'request_id': 'req-9542bf7d-b064-4430-aa40-015ef47d5e15'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.344525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:17.353583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b59e674e34f4f9cbc1619e2f4c0ad9e', 'request_id': 'req-9542bf7d-b064-4430-aa40-015ef47d5e15'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.365330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b59e674e34f4f9cbc1619e2f4c0ad9e', 'request_id': 'req-9542bf7d-b064-4430-aa40-015ef47d5e15'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.375438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] The token cache is being invalidate because project 4b59e674e34f4f9cbc1619e2f4c0ad9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:17.375730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.376401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.379285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 930/1862] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:17 2022] DELETE /identity/v3/projects/4b59e674e34f4f9cbc1619e2f4c0ad9e => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:17.382432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.384598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:17.406501 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 933/1863] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:17.407489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d0b6cf-506a-4721-8259-5eb5b944bb50 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.408785 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-50d0b6cf-506a-4721-8259-5eb5b944bb50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:17.450998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9542bf7d-b064-4430-aa40-015ef47d5e15 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.455792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bfb4f-4b9f-4ec6-b5ba-c956b1ca0938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.456459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bfb4f-4b9f-4ec6-b5ba-c956b1ca0938 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.456741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bfb4f-4b9f-4ec6-b5ba-c956b1ca0938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.457017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37bfb4f-4b9f-4ec6-b5ba-c956b1ca0938 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.457774 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bfb4f-4b9f-4ec6-b5ba-c956b1ca0938 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:17.458081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bfb4f-4b9f-4ec6-b5ba-c956b1ca0938 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.460252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37bfb4f-4b9f-4ec6-b5ba-c956b1ca0938 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.472744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 931/1864] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:17.473684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bfb4f-4b9f-4ec6-b5ba-c956b1ca0938 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.477468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e37bfb4f-4b9f-4ec6-b5ba-c956b1ca0938 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:17.477745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d0b6cf-506a-4721-8259-5eb5b944bb50 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.482421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.483304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.483588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.483870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] PATH_INFO: `/v3/users/3cd07cb936424e578bed7695432434ea` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.485080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cd07cb936424e578bed7695432434ea)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.487056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.490851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37bfb4f-4b9f-4ec6-b5ba-c956b1ca0938 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.496039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf96277-8fc3-4ff7-b385-dd7668eec5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.497273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf96277-8fc3-4ff7-b385-dd7668eec5b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.497273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf96277-8fc3-4ff7-b385-dd7668eec5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.497273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf96277-8fc3-4ff7-b385-dd7668eec5b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.498045 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf96277-8fc3-4ff7-b385-dd7668eec5b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.500306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf96277-8fc3-4ff7-b385-dd7668eec5b3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.524460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:17.537808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 932/1865] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:17 2022] POST /identity/v3/projects => generated 381 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:17.537808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf96277-8fc3-4ff7-b385-dd7668eec5b3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.545288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cd07cb936424e578bed7695432434ea'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.558002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cd07cb936424e578bed7695432434ea'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.564976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cd07cb936424e578bed7695432434ea'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.565753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cd07cb936424e578bed7695432434ea'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.566382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cd07cb936424e578bed7695432434ea'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:17.582475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 934/1866] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:17 2022] DELETE /identity/v3/users/3cd07cb936424e578bed7695432434ea => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:17.583384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.586325 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:17.608468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf96277-8fc3-4ff7-b385-dd7668eec5b3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.634289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:17.671680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799b6ad5-2c37-4d9a-89ec-baac6f8ab0d3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.680080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750333dd-caf7-4d5e-ab0b-df85e7857b4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.681146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750333dd-caf7-4d5e-ab0b-df85e7857b4a admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.681558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750333dd-caf7-4d5e-ab0b-df85e7857b4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.681801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750333dd-caf7-4d5e-ab0b-df85e7857b4a admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.682634 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750333dd-caf7-4d5e-ab0b-df85e7857b4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.685054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750333dd-caf7-4d5e-ab0b-df85e7857b4a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.688334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46871760-f28b-4a7c-9644-14924a28dde4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.689732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46871760-f28b-4a7c-9644-14924a28dde4 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.690244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46871760-f28b-4a7c-9644-14924a28dde4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.690671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46871760-f28b-4a7c-9644-14924a28dde4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.691529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46871760-f28b-4a7c-9644-14924a28dde4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.693888 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46871760-f28b-4a7c-9644-14924a28dde4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.712290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 935/1867] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:17 2022] POST /identity/v3/users => generated 297 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:17.718580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750333dd-caf7-4d5e-ab0b-df85e7857b4a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.735437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 933/1868] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:17 2022] POST /identity/v3/projects => generated 355 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:17.738830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750333dd-caf7-4d5e-ab0b-df85e7857b4a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.746519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46871760-f28b-4a7c-9644-14924a28dde4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.749433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-46871760-f28b-4a7c-9644-14924a28dde4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:17.848788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46871760-f28b-4a7c-9644-14924a28dde4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.860307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209d442-d236-44d5-9c58-3604d308290b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.861097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209d442-d236-44d5-9c58-3604d308290b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.861392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209d442-d236-44d5-9c58-3604d308290b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.861879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4209d442-d236-44d5-9c58-3604d308290b admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.864165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209d442-d236-44d5-9c58-3604d308290b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.866382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4209d442-d236-44d5-9c58-3604d308290b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.871971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0129a0-63f6-4d34-a258-c22db3523d74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.872825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0129a0-63f6-4d34-a258-c22db3523d74 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.873222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0129a0-63f6-4d34-a258-c22db3523d74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.873740 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0129a0-63f6-4d34-a258-c22db3523d74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.874623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0129a0-63f6-4d34-a258-c22db3523d74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.876951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0129a0-63f6-4d34-a258-c22db3523d74 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.893995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 936/1869] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:17.894996 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0129a0-63f6-4d34-a258-c22db3523d74 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.903396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 934/1870] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:17 2022] POST /identity/v3/users => generated 278 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:17.907873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209d442-d236-44d5-9c58-3604d308290b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.913947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0129a0-63f6-4d34-a258-c22db3523d74 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.923595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936b4c3-13c2-4671-8844-127f4fef7fe7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.924699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936b4c3-13c2-4671-8844-127f4fef7fe7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.924983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936b4c3-13c2-4671-8844-127f4fef7fe7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.925347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9936b4c3-13c2-4671-8844-127f4fef7fe7 admin admin] PATH_INFO: `/v3/projects/623f37ceaee641a8a2bd50972afabaff/users/149eae66cd9b48da9941ae248150e97c/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.930497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4209d442-d236-44d5-9c58-3604d308290b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:17.950916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9fbf1-259a-4817-8202-0318b81ae3ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:17.951751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9fbf1-259a-4817-8202-0318b81ae3ab admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:17.951986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9fbf1-259a-4817-8202-0318b81ae3ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:17.952229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb9fbf1-259a-4817-8202-0318b81ae3ab admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:17.954348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9fbf1-259a-4817-8202-0318b81ae3ab admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.957798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb9fbf1-259a-4817-8202-0318b81ae3ab admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.966764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936b4c3-13c2-4671-8844-127f4fef7fe7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=623f37ceaee641a8a2bd50972afabaff, user_id=149eae66cd9b48da9941ae248150e97c, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:17.973589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9936b4c3-13c2-4671-8844-127f4fef7fe7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:17.987920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 935/1871] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:17.989947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9fbf1-259a-4817-8202-0318b81ae3ab admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:17.999612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 937/1872] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:17 2022] PUT /identity/v3/projects/623f37ceaee641a8a2bd50972afabaff/users/149eae66cd9b48da9941ae248150e97c/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:18.021840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936b4c3-13c2-4671-8844-127f4fef7fe7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:18.022103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb9fbf1-259a-4817-8202-0318b81ae3ab admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:18.039156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:18.127446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fe48b-d43b-4410-b845-c37fbf6277af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:18.129714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fe48b-d43b-4410-b845-c37fbf6277af admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.130013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fe48b-d43b-4410-b845-c37fbf6277af admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.130294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70fe48b-d43b-4410-b845-c37fbf6277af admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.131044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fe48b-d43b-4410-b845-c37fbf6277af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:18.134272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70fe48b-d43b-4410-b845-c37fbf6277af admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:18.175574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 936/1873] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:17 2022] GET /identity/v3/roles => generated 1420 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:18.177208 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fe48b-d43b-4410-b845-c37fbf6277af admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:18.186018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f70fe48b-d43b-4410-b845-c37fbf6277af admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.206770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70fe48b-d43b-4410-b845-c37fbf6277af admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:18.212259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:18.213257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.213588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.213909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] PATH_INFO: `/v3/projects/03d63ab890ae488b8d0673b9dcb458c1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.215522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] RBAC: Authorizing `identity:delete_project(project_id=03d63ab890ae488b8d0673b9dcb458c1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:18.218343 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:18.219129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9936b4c3-13c2-4671-8844-127f4fef7fe7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:18.232955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fa657-5cf4-43a7-a8b8-76a4f48b816c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:18.233769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fa657-5cf4-43a7-a8b8-76a4f48b816c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.233980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fa657-5cf4-43a7-a8b8-76a4f48b816c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.234245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fa657-5cf4-43a7-a8b8-76a4f48b816c admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.235162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fa657-5cf4-43a7-a8b8-76a4f48b816c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:18.239063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fa657-5cf4-43a7-a8b8-76a4f48b816c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:18.282276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 938/1874] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:18 2022] GET /identity/v3/roles => generated 1420 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:18.282276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fa657-5cf4-43a7-a8b8-76a4f48b816c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:18.303976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fa657-5cf4-43a7-a8b8-76a4f48b816c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:18.311391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '03d63ab890ae488b8d0673b9dcb458c1', 'request_id': 'req-edf4ca36-e20c-420d-b9a6-db4a044eb041'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:18.319353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '03d63ab890ae488b8d0673b9dcb458c1', 'request_id': 'req-edf4ca36-e20c-420d-b9a6-db4a044eb041'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:18.326283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '03d63ab890ae488b8d0673b9dcb458c1', 'request_id': 'req-edf4ca36-e20c-420d-b9a6-db4a044eb041'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:18.349769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] The token cache is being invalidate because project 03d63ab890ae488b8d0673b9dcb458c1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:18.350910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:18.353118 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:18.359395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 937/1875] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:18 2022] DELETE /identity/v3/projects/03d63ab890ae488b8d0673b9dcb458c1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:18.360463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:18.370207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.376472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:18.416434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b18305-a324-4af0-a951-5845730706e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:18.417412 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b18305-a324-4af0-a951-5845730706e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.417720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b18305-a324-4af0-a951-5845730706e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.417994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b18305-a324-4af0-a951-5845730706e7 admin admin] PATH_INFO: `/v3/projects/47775f27e33448059d70a822b0931022/users/30a9be9e06e54718bc157486848fceca/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.442635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b18305-a324-4af0-a951-5845730706e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=47775f27e33448059d70a822b0931022, user_id=30a9be9e06e54718bc157486848fceca, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:18.446259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b18305-a324-4af0-a951-5845730706e7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:18.469431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 939/1876] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:18 2022] PUT /identity/v3/projects/47775f27e33448059d70a822b0931022/users/30a9be9e06e54718bc157486848fceca/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:18.471375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3276714-2408-42e8-9cc4-76c6d0272af0 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.474437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3276714-2408-42e8-9cc4-76c6d0272af0 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.474998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3276714-2408-42e8-9cc4-76c6d0272af0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.507616 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c3276714-2408-42e8-9cc4-76c6d0272af0 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:18.508856 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c3276714-2408-42e8-9cc4-76c6d0272af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.512585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edf4ca36-e20c-420d-b9a6-db4a044eb041 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:18.559491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:18.611726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c775037-ff84-4934-bfec-99f501ac8981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:18.612825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775037-ff84-4934-bfec-99f501ac8981 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.613254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775037-ff84-4934-bfec-99f501ac8981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.613731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c775037-ff84-4934-bfec-99f501ac8981 admin admin] PATH_INFO: `/v3/projects/623f37ceaee641a8a2bd50972afabaff/users/149eae66cd9b48da9941ae248150e97c/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.616751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c775037-ff84-4934-bfec-99f501ac8981 admin admin] RBAC: Authorizing `identity:create_grant(project_id=623f37ceaee641a8a2bd50972afabaff, user_id=149eae66cd9b48da9941ae248150e97c, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:18.620666 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c775037-ff84-4934-bfec-99f501ac8981 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:18.632523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 940/1877] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:18.632523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d983e40-5660-45e8-a812-f8b702912d3f None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.632523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d983e40-5660-45e8-a812-f8b702912d3f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.632523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d983e40-5660-45e8-a812-f8b702912d3f None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.632523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 941/1878] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:24:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:24:18.633420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe6c55-89af-48dd-a87c-86d20407d0fd None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.634309 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe6c55-89af-48dd-a87c-86d20407d0fd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.634835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fe6c55-89af-48dd-a87c-86d20407d0fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.642121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 938/1879] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:18 2022] PUT /identity/v3/projects/623f37ceaee641a8a2bd50972afabaff/users/149eae66cd9b48da9941ae248150e97c/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 282 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:18.643835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4d38f-65ea-4fa2-b488-5b4d4e69a299 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.644309 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4d38f-65ea-4fa2-b488-5b4d4e69a299 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.644788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d4d38f-65ea-4fa2-b488-5b4d4e69a299 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.675906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-92d4d38f-65ea-4fa2-b488-5b4d4e69a299 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:18.677481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-92d4d38f-65ea-4fa2-b488-5b4d4e69a299 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.703122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-39fe6c55-89af-48dd-a87c-86d20407d0fd None None] MFA Rules not processed for user `30a9be9e06e54718bc157486848fceca`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:18.706378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-39fe6c55-89af-48dd-a87c-86d20407d0fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.771168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 939/1880] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:18.776700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cfafe-fe74-4793-966e-16abe91b34bf None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.777568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cfafe-fe74-4793-966e-16abe91b34bf None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.778028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491cfafe-fe74-4793-966e-16abe91b34bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.806206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-491cfafe-fe74-4793-966e-16abe91b34bf None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:18.808490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-491cfafe-fe74-4793-966e-16abe91b34bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.814504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 942/1881] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:18.815884 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63c9eb-4711-48d0-aff2-b53e3a65dca9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.816175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63c9eb-4711-48d0-aff2-b53e3a65dca9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.816454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d63c9eb-4711-48d0-aff2-b53e3a65dca9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.869291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8d63c9eb-4711-48d0-aff2-b53e3a65dca9 None None] MFA Rules not processed for user `149eae66cd9b48da9941ae248150e97c`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:18.871886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8d63c9eb-4711-48d0-aff2-b53e3a65dca9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.916019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 940/1882] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:18.917038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cfafe-fe74-4793-966e-16abe91b34bf None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:18.939768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:18.945162 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491cfafe-fe74-4793-966e-16abe91b34bf None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:18.952352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08133a79-cac7-470a-a4db-4cf4c9aa72aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:18.953243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08133a79-cac7-470a-a4db-4cf4c9aa72aa admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:18.953549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08133a79-cac7-470a-a4db-4cf4c9aa72aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:18.953828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08133a79-cac7-470a-a4db-4cf4c9aa72aa admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:18.954559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08133a79-cac7-470a-a4db-4cf4c9aa72aa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:18.954867 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08133a79-cac7-470a-a4db-4cf4c9aa72aa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:18.956755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08133a79-cac7-470a-a4db-4cf4c9aa72aa admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:18.972304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 941/1883] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:18.973351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08133a79-cac7-470a-a4db-4cf4c9aa72aa admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:18.976239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-08133a79-cac7-470a-a4db-4cf4c9aa72aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:18.990736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 943/1884] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:18 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:18.992104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63c9eb-4711-48d0-aff2-b53e3a65dca9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.002783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08133a79-cac7-470a-a4db-4cf4c9aa72aa admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.007066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d63c9eb-4711-48d0-aff2-b53e3a65dca9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.010916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9a077-52f7-4e33-b310-843531c9f4ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.011909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a077-52f7-4e33-b310-843531c9f4ec admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.012480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a077-52f7-4e33-b310-843531c9f4ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.012817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10923ff-98c3-4a9a-a109-ac37a4c4c8d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.013183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10d9a077-52f7-4e33-b310-843531c9f4ec admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.013757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10923ff-98c3-4a9a-a109-ac37a4c4c8d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.014246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10923ff-98c3-4a9a-a109-ac37a4c4c8d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.014374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9a077-52f7-4e33-b310-843531c9f4ec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.015621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10923ff-98c3-4a9a-a109-ac37a4c4c8d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.016640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10923ff-98c3-4a9a-a109-ac37a4c4c8d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.018129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10d9a077-52f7-4e33-b310-843531c9f4ec admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.018786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10923ff-98c3-4a9a-a109-ac37a4c4c8d8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.074405 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 944/1886] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:18 2022] POST /identity/v3/projects => generated 379 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:19.074595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 942/1886] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:18 2022] POST /identity/v3/projects => generated 355 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:19.074702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10923ff-98c3-4a9a-a109-ac37a4c4c8d8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.074780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9a077-52f7-4e33-b310-843531c9f4ec admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.074850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b10923ff-98c3-4a9a-a109-ac37a4c4c8d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:19.074935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-10d9a077-52f7-4e33-b310-843531c9f4ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:19.131697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:19.159860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10923ff-98c3-4a9a-a109-ac37a4c4c8d8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.245095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10d9a077-52f7-4e33-b310-843531c9f4ec admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.254661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c899e80-1b5c-456c-ac7e-351709b97754 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.256444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c899e80-1b5c-456c-ac7e-351709b97754 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.258011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c899e80-1b5c-456c-ac7e-351709b97754 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.258301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c899e80-1b5c-456c-ac7e-351709b97754 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.260592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59e1de-992e-4cfc-9db6-df822d8e46b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.261684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59e1de-992e-4cfc-9db6-df822d8e46b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.262332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59e1de-992e-4cfc-9db6-df822d8e46b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.262832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae59e1de-992e-4cfc-9db6-df822d8e46b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.271316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c899e80-1b5c-456c-ac7e-351709b97754 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.283659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c899e80-1b5c-456c-ac7e-351709b97754 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.294603 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 943/1887] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:19 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:19.295291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c899e80-1b5c-456c-ac7e-351709b97754 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.405728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c899e80-1b5c-456c-ac7e-351709b97754 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.411266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59e1de-992e-4cfc-9db6-df822d8e46b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.412853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae59e1de-992e-4cfc-9db6-df822d8e46b4 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.424126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59637af0-5cc8-45f1-b43d-77b73b962ab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.424811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59637af0-5cc8-45f1-b43d-77b73b962ab3 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.425781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59637af0-5cc8-45f1-b43d-77b73b962ab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.426064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59637af0-5cc8-45f1-b43d-77b73b962ab3 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.429626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59637af0-5cc8-45f1-b43d-77b73b962ab3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.431829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59637af0-5cc8-45f1-b43d-77b73b962ab3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.436916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 945/1888] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:19 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:19.438214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59e1de-992e-4cfc-9db6-df822d8e46b4 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.458796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae59e1de-992e-4cfc-9db6-df822d8e46b4 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.473106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40290ae8-3fc0-4065-96ba-6c702a79a5fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.475082 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40290ae8-3fc0-4065-96ba-6c702a79a5fc admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.475530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40290ae8-3fc0-4065-96ba-6c702a79a5fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.479368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 944/1889] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:19 2022] POST /identity/v3/users => generated 284 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:19.479521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40290ae8-3fc0-4065-96ba-6c702a79a5fc admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.479521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40290ae8-3fc0-4065-96ba-6c702a79a5fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.481009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59637af0-5cc8-45f1-b43d-77b73b962ab3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.481965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40290ae8-3fc0-4065-96ba-6c702a79a5fc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.505095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59637af0-5cc8-45f1-b43d-77b73b962ab3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.523411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:19.527336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38437022-2e4e-410e-864c-8dbdb2b5f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.528539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38437022-2e4e-410e-864c-8dbdb2b5f2d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.531453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38437022-2e4e-410e-864c-8dbdb2b5f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.532845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38437022-2e4e-410e-864c-8dbdb2b5f2d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.533432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 946/1890] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:19 2022] POST /identity/v3/users => generated 290 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:19.534702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40290ae8-3fc0-4065-96ba-6c702a79a5fc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.539289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38437022-2e4e-410e-864c-8dbdb2b5f2d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.547594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40290ae8-3fc0-4065-96ba-6c702a79a5fc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.556796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b40be-d2a4-4a59-a956-bf674ab8e4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.556796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b40be-d2a4-4a59-a956-bf674ab8e4b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.556796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b40be-d2a4-4a59-a956-bf674ab8e4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.557166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38437022-2e4e-410e-864c-8dbdb2b5f2d2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.557272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-906b40be-d2a4-4a59-a956-bf674ab8e4b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.557272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b40be-d2a4-4a59-a956-bf674ab8e4b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.557272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-906b40be-d2a4-4a59-a956-bf674ab8e4b0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.572537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 947/1891] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:19.573468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b40be-d2a4-4a59-a956-bf674ab8e4b0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.586052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-906b40be-d2a4-4a59-a956-bf674ab8e4b0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.591296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84c1cb-d715-4647-be7a-04a17b90e02f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.593085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84c1cb-d715-4647-be7a-04a17b90e02f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.593268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84c1cb-d715-4647-be7a-04a17b90e02f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.593612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa84c1cb-d715-4647-be7a-04a17b90e02f admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.594409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84c1cb-d715-4647-be7a-04a17b90e02f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.596316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa84c1cb-d715-4647-be7a-04a17b90e02f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.602102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 945/1892] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 122 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:19.607762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38437022-2e4e-410e-864c-8dbdb2b5f2d2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.613995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 948/1893] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:19.614908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84c1cb-d715-4647-be7a-04a17b90e02f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.626443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa84c1cb-d715-4647-be7a-04a17b90e02f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.630182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38437022-2e4e-410e-864c-8dbdb2b5f2d2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.633767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8328627-b9b4-45c4-bcbc-8379b44a3c57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.634813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8328627-b9b4-45c4-bcbc-8379b44a3c57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.635376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8328627-b9b4-45c4-bcbc-8379b44a3c57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.635808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8328627-b9b4-45c4-bcbc-8379b44a3c57 admin admin] PATH_INFO: `/v3/projects/6d1b7b1c0cb149a0a0097de2bb286fde/users/7c2a9ed4780d486f8482876c4a78dda6/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.647872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ddeb2-a162-4ba1-86c1-6a94107a5371 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.649673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ddeb2-a162-4ba1-86c1-6a94107a5371 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.650314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ddeb2-a162-4ba1-86c1-6a94107a5371 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.650832 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931ddeb2-a162-4ba1-86c1-6a94107a5371 admin admin] PATH_INFO: `/v3/projects/e839a5f0a3884f5ab6e3ba92587b7117/users/5a1da32382cd484795961dc4f0d47e49/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.655885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8328627-b9b4-45c4-bcbc-8379b44a3c57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6d1b7b1c0cb149a0a0097de2bb286fde, user_id=7c2a9ed4780d486f8482876c4a78dda6, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.659466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8328627-b9b4-45c4-bcbc-8379b44a3c57 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.691743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 949/1894] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:19 2022] PUT /identity/v3/projects/6d1b7b1c0cb149a0a0097de2bb286fde/users/7c2a9ed4780d486f8482876c4a78dda6/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:19.693389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed70d2-636e-4ccb-baee-caf57f0e63b1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.693677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ddeb2-a162-4ba1-86c1-6a94107a5371 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e839a5f0a3884f5ab6e3ba92587b7117, user_id=5a1da32382cd484795961dc4f0d47e49, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.693855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed70d2-636e-4ccb-baee-caf57f0e63b1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.694150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48ed70d2-636e-4ccb-baee-caf57f0e63b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.701856 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931ddeb2-a162-4ba1-86c1-6a94107a5371 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.725504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-48ed70d2-636e-4ccb-baee-caf57f0e63b1 None None] MFA Rules not processed for user `7c2a9ed4780d486f8482876c4a78dda6`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:19.726803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-48ed70d2-636e-4ccb-baee-caf57f0e63b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:19.730279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 946/1895] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:19 2022] PUT /identity/v3/projects/e839a5f0a3884f5ab6e3ba92587b7117/users/5a1da32382cd484795961dc4f0d47e49/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:19.734074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ddeb2-a162-4ba1-86c1-6a94107a5371 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.811178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 950/1896] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:19 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:19.812190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed70d2-636e-4ccb-baee-caf57f0e63b1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.888207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48ed70d2-636e-4ccb-baee-caf57f0e63b1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.895620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ab7c4-05f7-4900-92a4-08e4feb86712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.896884 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab7c4-05f7-4900-92a4-08e4feb86712 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.897381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab7c4-05f7-4900-92a4-08e4feb86712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.898072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60ab7c4-05f7-4900-92a4-08e4feb86712 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.898985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ab7c4-05f7-4900-92a4-08e4feb86712 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.901203 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60ab7c4-05f7-4900-92a4-08e4feb86712 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.917614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931ddeb2-a162-4ba1-86c1-6a94107a5371 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.923479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad92444-ba02-4c0c-9eae-8cc681b9b9d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.924388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad92444-ba02-4c0c-9eae-8cc681b9b9d0 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.924914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad92444-ba02-4c0c-9eae-8cc681b9b9d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.925416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad92444-ba02-4c0c-9eae-8cc681b9b9d0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.926456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad92444-ba02-4c0c-9eae-8cc681b9b9d0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.929848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad92444-ba02-4c0c-9eae-8cc681b9b9d0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:19.948509 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 947/1897] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:19 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:19.952352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad92444-ba02-4c0c-9eae-8cc681b9b9d0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:19.963406 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:19.970043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad92444-ba02-4c0c-9eae-8cc681b9b9d0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:19.988966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e820b202-450e-41d0-b98c-9aa881168063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:19.990061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820b202-450e-41d0-b98c-9aa881168063 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:19.991693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820b202-450e-41d0-b98c-9aa881168063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:19.991693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e820b202-450e-41d0-b98c-9aa881168063 admin admin] PATH_INFO: `/v3/projects/e839a5f0a3884f5ab6e3ba92587b7117/users/5a1da32382cd484795961dc4f0d47e49/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:19.995930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820b202-450e-41d0-b98c-9aa881168063 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e839a5f0a3884f5ab6e3ba92587b7117, user_id=5a1da32382cd484795961dc4f0d47e49, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:19.998381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 951/1898] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:19 2022] POST /identity/v3/projects => generated 377 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:19.999892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ab7c4-05f7-4900-92a4-08e4feb86712 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.005577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e820b202-450e-41d0-b98c-9aa881168063 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.061654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 948/1899] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:19 2022] PUT /identity/v3/projects/e839a5f0a3884f5ab6e3ba92587b7117/users/5a1da32382cd484795961dc4f0d47e49/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:20.065780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de48be-7617-48f2-94c8-ba223bbe472a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.066080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de48be-7617-48f2-94c8-ba223bbe472a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.066367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20de48be-7617-48f2-94c8-ba223bbe472a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.077882 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:20.105399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-20de48be-7617-48f2-94c8-ba223bbe472a None None] MFA Rules not processed for user `5a1da32382cd484795961dc4f0d47e49`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:20.106900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-20de48be-7617-48f2-94c8-ba223bbe472a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:20.161279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60ab7c4-05f7-4900-92a4-08e4feb86712 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.258249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd4844-90c8-4bca-a6e1-9ed5c1453e98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:20.261167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd4844-90c8-4bca-a6e1-9ed5c1453e98 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.263547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd4844-90c8-4bca-a6e1-9ed5c1453e98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.263547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afd4844-90c8-4bca-a6e1-9ed5c1453e98 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.263547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd4844-90c8-4bca-a6e1-9ed5c1453e98 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:20.267064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afd4844-90c8-4bca-a6e1-9ed5c1453e98 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.297223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 949/1900] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:20 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:20.344251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 952/1901] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:19 2022] POST /identity/v3/users => generated 295 bytes in 345 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:20.347018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de48be-7617-48f2-94c8-ba223bbe472a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.358484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd4844-90c8-4bca-a6e1-9ed5c1453e98 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.399635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afd4844-90c8-4bca-a6e1-9ed5c1453e98 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.411749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e1d26-3c89-42dc-af81-9347b79822fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:20.413032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e1d26-3c89-42dc-af81-9347b79822fa admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.413309 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e1d26-3c89-42dc-af81-9347b79822fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.413613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e1d26-3c89-42dc-af81-9347b79822fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.414365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e1d26-3c89-42dc-af81-9347b79822fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:20.425465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540e1d26-3c89-42dc-af81-9347b79822fa admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.463574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 953/1902] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:20.463574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e1d26-3c89-42dc-af81-9347b79822fa admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.486728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540e1d26-3c89-42dc-af81-9347b79822fa admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.499685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:20.500767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.501043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20de48be-7617-48f2-94c8-ba223bbe472a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.502217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.502562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] PATH_INFO: `/v3/users/7923d8bc001b49e28518dad0bafaa002` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.506910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7923d8bc001b49e28518dad0bafaa002)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:20.509552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fd1d5-da87-442d-877a-0cd396dc7ab4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:20.510946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fd1d5-da87-442d-877a-0cd396dc7ab4 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.511522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fd1d5-da87-442d-877a-0cd396dc7ab4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.512078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1fd1d5-da87-442d-877a-0cd396dc7ab4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.513325 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.533692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1fd1d5-da87-442d-877a-0cd396dc7ab4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:20.536444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e1fd1d5-da87-442d-877a-0cd396dc7ab4 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.560630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 950/1903] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:20 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 214 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:20.562254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fd1d5-da87-442d-877a-0cd396dc7ab4 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.577567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1fd1d5-da87-442d-877a-0cd396dc7ab4 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.582559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778cc95-1d59-446e-bc25-83e3443bcc55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:20.583515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778cc95-1d59-446e-bc25-83e3443bcc55 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.583798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778cc95-1d59-446e-bc25-83e3443bcc55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.584202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5778cc95-1d59-446e-bc25-83e3443bcc55 admin admin] PATH_INFO: `/v3/projects/689a18fa26f04804b4633df4b2bda599/users/196d45883d23430695dd63b16c1a6693/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.607957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778cc95-1d59-446e-bc25-83e3443bcc55 admin admin] RBAC: Authorizing `identity:create_grant(project_id=689a18fa26f04804b4633df4b2bda599, user_id=196d45883d23430695dd63b16c1a6693, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:20.618504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5778cc95-1d59-446e-bc25-83e3443bcc55 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:20.653552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 951/1904] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:20 2022] PUT /identity/v3/projects/689a18fa26f04804b4633df4b2bda599/users/196d45883d23430695dd63b16c1a6693/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:20.657781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778cc95-1d59-446e-bc25-83e3443bcc55 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.658492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7923d8bc001b49e28518dad0bafaa002'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:20.688631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7923d8bc001b49e28518dad0bafaa002'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:20.714953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7923d8bc001b49e28518dad0bafaa002'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:20.716031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7923d8bc001b49e28518dad0bafaa002'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:20.716779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-631f962f-d389-4aa9-a6ea-6a2aced40c81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7923d8bc001b49e28518dad0bafaa002'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:20.750248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 954/1905] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:20 2022] DELETE /identity/v3/users/7923d8bc001b49e28518dad0bafaa002 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:20.757629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35069cd-285b-4d29-8730-8597e970dfd6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:20.758126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35069cd-285b-4d29-8730-8597e970dfd6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:20.758485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35069cd-285b-4d29-8730-8597e970dfd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:20.801067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a35069cd-285b-4d29-8730-8597e970dfd6 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:20.802483 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a35069cd-285b-4d29-8730-8597e970dfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:20.834794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:20.848269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5778cc95-1d59-446e-bc25-83e3443bcc55 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:20.856097 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:20.960810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 955/1906] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:20.960810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35069cd-285b-4d29-8730-8597e970dfd6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:20.960810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a35069cd-285b-4d29-8730-8597e970dfd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.015391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621fac2-8560-4c3c-b6ab-c47efe885906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.016124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621fac2-8560-4c3c-b6ab-c47efe885906 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.016414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621fac2-8560-4c3c-b6ab-c47efe885906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.016695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621fac2-8560-4c3c-b6ab-c47efe885906 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.017433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621fac2-8560-4c3c-b6ab-c47efe885906 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.019123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621fac2-8560-4c3c-b6ab-c47efe885906 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.030507 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 952/1907] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:20 2022] GET /identity/v3/roles => generated 1420 bytes in 373 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:21.031747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621fac2-8560-4c3c-b6ab-c47efe885906 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.034243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0621fac2-8560-4c3c-b6ab-c47efe885906 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.069935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35069cd-285b-4d29-8730-8597e970dfd6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.074671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea1dff60-e895-4bae-83a4-7b98d82dd7a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.075318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dff60-e895-4bae-83a4-7b98d82dd7a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.075752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dff60-e895-4bae-83a4-7b98d82dd7a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.076149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea1dff60-e895-4bae-83a4-7b98d82dd7a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.079433 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ea1dff60-e895-4bae-83a4-7b98d82dd7a3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.107673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621fac2-8560-4c3c-b6ab-c47efe885906 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.112734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515601f9-5fbd-46cf-af05-9fa13d4e758d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.113253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515601f9-5fbd-46cf-af05-9fa13d4e758d service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.113622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515601f9-5fbd-46cf-af05-9fa13d4e758d service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.113898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-515601f9-5fbd-46cf-af05-9fa13d4e758d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.126763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515601f9-5fbd-46cf-af05-9fa13d4e758d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.127298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-515601f9-5fbd-46cf-af05-9fa13d4e758d service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.139491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 953/1908] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:21.140134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515601f9-5fbd-46cf-af05-9fa13d4e758d service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.141416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-515601f9-5fbd-46cf-af05-9fa13d4e758d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.148728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dff60-e895-4bae-83a4-7b98d82dd7a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.150823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea1dff60-e895-4bae-83a4-7b98d82dd7a3 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.157222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-515601f9-5fbd-46cf-af05-9fa13d4e758d service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.165360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92476990-7665-4233-a630-c4e6034457d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.166434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476990-7665-4233-a630-c4e6034457d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.166943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476990-7665-4233-a630-c4e6034457d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.167184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92476990-7665-4233-a630-c4e6034457d2 admin admin] PATH_INFO: `/v3/projects/689a18fa26f04804b4633df4b2bda599/users/196d45883d23430695dd63b16c1a6693/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.170005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92476990-7665-4233-a630-c4e6034457d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=689a18fa26f04804b4633df4b2bda599, user_id=196d45883d23430695dd63b16c1a6693, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.171708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 956/1909] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:20 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:21.173136 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92476990-7665-4233-a630-c4e6034457d2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.191401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 954/1910] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:21 2022] PUT /identity/v3/projects/689a18fa26f04804b4633df4b2bda599/users/196d45883d23430695dd63b16c1a6693/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:21.208486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d5b8-0236-4a74-aff6-5513641a2eda None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.208788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d5b8-0236-4a74-aff6-5513641a2eda None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.209117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33c6d5b8-0236-4a74-aff6-5513641a2eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.254175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-33c6d5b8-0236-4a74-aff6-5513641a2eda None None] MFA Rules not processed for user `196d45883d23430695dd63b16c1a6693`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:21.256628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-33c6d5b8-0236-4a74-aff6-5513641a2eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.401550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 957/1911] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:21 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:21.413822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92476990-7665-4233-a630-c4e6034457d2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.498193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92476990-7665-4233-a630-c4e6034457d2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.501866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c1c63-912c-4221-aa1a-3e06dfd40c1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.502565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c1c63-912c-4221-aa1a-3e06dfd40c1e service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.502797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c1c63-912c-4221-aa1a-3e06dfd40c1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.503087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e70c1c63-912c-4221-aa1a-3e06dfd40c1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.512873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c1c63-912c-4221-aa1a-3e06dfd40c1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.514338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e70c1c63-912c-4221-aa1a-3e06dfd40c1e service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.526459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 955/1912] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:21 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:21.727347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d5b8-0236-4a74-aff6-5513641a2eda None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.729702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-33c6d5b8-0236-4a74-aff6-5513641a2eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.739915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:21.820815 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33c6d5b8-0236-4a74-aff6-5513641a2eda None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.833694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:21.834766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:21.835256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:21.835853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] PATH_INFO: `/v3/projects/5dd491b6a7da44d68f496d142fce9267` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:21.837468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5dd491b6a7da44d68f496d142fce9267)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:21.839457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:21.918546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5dd491b6a7da44d68f496d142fce9267', 'request_id': 'req-420eafee-b8db-4e6c-9469-286d29dbb9c3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:21.922761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c1c63-912c-4221-aa1a-3e06dfd40c1e service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.927767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5dd491b6a7da44d68f496d142fce9267', 'request_id': 'req-420eafee-b8db-4e6c-9469-286d29dbb9c3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:21.933629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e70c1c63-912c-4221-aa1a-3e06dfd40c1e service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:21.936941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5dd491b6a7da44d68f496d142fce9267', 'request_id': 'req-420eafee-b8db-4e6c-9469-286d29dbb9c3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:21.952962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] The token cache is being invalidate because project 5dd491b6a7da44d68f496d142fce9267 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:21.953151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:21.953839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:21.955409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 958/1913] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:21 2022] DELETE /identity/v3/projects/5dd491b6a7da44d68f496d142fce9267 => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:21.959389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:21.961200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:21.970759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:22.012054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c656f-c467-4dc1-8056-172e8a3d8704 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196d45883d23430695dd63b16c1a6693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '689a18fa26f04804b4633df4b2bda599', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.012726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c656f-c467-4dc1-8056-172e8a3d8704 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.013016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c656f-c467-4dc1-8056-172e8a3d8704 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.013296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132c656f-c467-4dc1-8056-172e8a3d8704 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.014061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c656f-c467-4dc1-8056-172e8a3d8704 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:22.014379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c656f-c467-4dc1-8056-172e8a3d8704 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.015982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-132c656f-c467-4dc1-8056-172e8a3d8704 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.027789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 956/1914] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:22.031901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c656f-c467-4dc1-8056-172e8a3d8704 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.033680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-132c656f-c467-4dc1-8056-172e8a3d8704 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.044044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420eafee-b8db-4e6c-9469-286d29dbb9c3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.048151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-132c656f-c467-4dc1-8056-172e8a3d8704 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.049170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.050083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.050499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.050785 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] PATH_INFO: `/v3/users/149eae66cd9b48da9941ae248150e97c` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.052024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] RBAC: Authorizing `identity:delete_user(user_id=149eae66cd9b48da9941ae248150e97c)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.052776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363c7ab-da6a-4944-bb16-eb33b7357384 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196d45883d23430695dd63b16c1a6693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '689a18fa26f04804b4633df4b2bda599', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.054055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363c7ab-da6a-4944-bb16-eb33b7357384 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.054445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363c7ab-da6a-4944-bb16-eb33b7357384 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.054727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9363c7ab-da6a-4944-bb16-eb33b7357384 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.054964 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.055417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363c7ab-da6a-4944-bb16-eb33b7357384 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.057299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9363c7ab-da6a-4944-bb16-eb33b7357384 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.093081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 957/1915] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:22 2022] POST /identity/v3/projects => generated 355 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:22.097776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363c7ab-da6a-4944-bb16-eb33b7357384 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.119140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:22.128454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '149eae66cd9b48da9941ae248150e97c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.148302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '149eae66cd9b48da9941ae248150e97c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.160080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '149eae66cd9b48da9941ae248150e97c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.161656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '149eae66cd9b48da9941ae248150e97c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.163009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '149eae66cd9b48da9941ae248150e97c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.184450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 959/1916] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:21 2022] DELETE /identity/v3/users/149eae66cd9b48da9941ae248150e97c => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:22.192552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9363c7ab-da6a-4944-bb16-eb33b7357384 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.277874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45054a39-732a-4e4a-888d-cf78c3df7962 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196d45883d23430695dd63b16c1a6693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '689a18fa26f04804b4633df4b2bda599', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.279051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45054a39-732a-4e4a-888d-cf78c3df7962 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.279548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45054a39-732a-4e4a-888d-cf78c3df7962 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.279875 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45054a39-732a-4e4a-888d-cf78c3df7962 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.280837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45054a39-732a-4e4a-888d-cf78c3df7962 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:22.282472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45054a39-732a-4e4a-888d-cf78c3df7962 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.283440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45054a39-732a-4e4a-888d-cf78c3df7962 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.309588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 958/1917] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:22.366771 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.368297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.394601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9100827d-18eb-40b6-9d39-c75aca1fb658 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.404106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196d45883d23430695dd63b16c1a6693', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '689a18fa26f04804b4633df4b2bda599', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.406930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.407292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.407643 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] PATH_INFO: `/v3/projects/55905f942c304514ae577e5f5e394931` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.412245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: Authorizing `identity:delete_project(project_id=55905f942c304514ae577e5f5e394931)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.415439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.524897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45054a39-732a-4e4a-888d-cf78c3df7962 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.526848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-45054a39-732a-4e4a-888d-cf78c3df7962 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.549029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55905f942c304514ae577e5f5e394931', 'request_id': 'req-f960ad73-6a77-4804-b22f-509b4f6a293b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.564564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55905f942c304514ae577e5f5e394931', 'request_id': 'req-f960ad73-6a77-4804-b22f-509b4f6a293b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.583633 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55905f942c304514ae577e5f5e394931', 'request_id': 'req-f960ad73-6a77-4804-b22f-509b4f6a293b'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.619236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:22.620068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] The token cache is being invalidate because project 55905f942c304514ae577e5f5e394931 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:22.620678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.622758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.629119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 960/1918] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:22 2022] DELETE /identity/v3/projects/55905f942c304514ae577e5f5e394931 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:22.697386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45054a39-732a-4e4a-888d-cf78c3df7962 tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.755328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:22.758363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:22.831566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.832520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.832883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.833502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] PATH_INFO: `/v3/users/d9bece2e8ca849db8fa9bfe853f57209` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.835062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9bece2e8ca849db8fa9bfe853f57209)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.837358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.866025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f960ad73-6a77-4804-b22f-509b4f6a293b tempest-VolumeQuotaClassesTest-14841126 tempest-VolumeQuotaClassesTest-14841126-project-admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:22.874532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:22.877705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.878008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.878293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] PATH_INFO: `/v3/projects/623f37ceaee641a8a2bd50972afabaff` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.881047 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] RBAC: Authorizing `identity:delete_project(project_id=623f37ceaee641a8a2bd50972afabaff)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:22.883191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:22.908658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9bece2e8ca849db8fa9bfe853f57209'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.924199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9bece2e8ca849db8fa9bfe853f57209'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.933808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9bece2e8ca849db8fa9bfe853f57209'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.935190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9bece2e8ca849db8fa9bfe853f57209'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.936217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b87bb54b-dbf0-40b5-9906-58c56780837f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9bece2e8ca849db8fa9bfe853f57209'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.947150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=15,keystone:SELECT=83,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:22.955767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '623f37ceaee641a8a2bd50972afabaff', 'request_id': 'req-67a085aa-415d-47b5-bc8b-22702f38ca2e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.961633 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 959/1919] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:22 2022] DELETE /identity/v3/users/d9bece2e8ca849db8fa9bfe853f57209 => generated 0 bytes in 437 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:22.964351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '623f37ceaee641a8a2bd50972afabaff', 'request_id': 'req-67a085aa-415d-47b5-bc8b-22702f38ca2e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.966379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18964829-01ff-42c0-894f-f8f73d52bfcc None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:22.966675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18964829-01ff-42c0-894f-f8f73d52bfcc None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:22.966960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18964829-01ff-42c0-894f-f8f73d52bfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:22.983685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '623f37ceaee641a8a2bd50972afabaff', 'request_id': 'req-67a085aa-415d-47b5-bc8b-22702f38ca2e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:22.987922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-18964829-01ff-42c0-894f-f8f73d52bfcc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:22.989490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-18964829-01ff-42c0-894f-f8f73d52bfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.002228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] The token cache is being invalidate because project 623f37ceaee641a8a2bd50972afabaff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:23.002583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:23.003256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-67a085aa-415d-47b5-bc8b-22702f38ca2e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:23.005329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 961/1920] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:22 2022] DELETE /identity/v3/projects/623f37ceaee641a8a2bd50972afabaff => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:23.013734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d1e36-5929-4d45-8b84-021e77651089 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.014031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d1e36-5929-4d45-8b84-021e77651089 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.014430 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52d1e36-5929-4d45-8b84-021e77651089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.036863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d52d1e36-5929-4d45-8b84-021e77651089 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:23.038601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d52d1e36-5929-4d45-8b84-021e77651089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.074946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 960/1921] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:23.075731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18964829-01ff-42c0-894f-f8f73d52bfcc None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.077055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-18964829-01ff-42c0-894f-f8f73d52bfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.091814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18964829-01ff-42c0-894f-f8f73d52bfcc None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.097504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.098420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.098735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.098978 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] PATH_INFO: `/v3/users/7c2a9ed4780d486f8482876c4a78dda6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.100425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c2a9ed4780d486f8482876c4a78dda6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.102394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.106130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:23.138032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 962/1922] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:23.139100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d1e36-5929-4d45-8b84-021e77651089 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.140397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d52d1e36-5929-4d45-8b84-021e77651089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.172779 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c2a9ed4780d486f8482876c4a78dda6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:23.188828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c2a9ed4780d486f8482876c4a78dda6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:23.196732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c2a9ed4780d486f8482876c4a78dda6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:23.197629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c2a9ed4780d486f8482876c4a78dda6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:23.198418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0b0de025-1cfb-4bfd-a924-6cdc1b199192 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c2a9ed4780d486f8482876c4a78dda6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:23.206268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52d1e36-5929-4d45-8b84-021e77651089 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.210628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58c5032-492c-4302-966f-0218cb41531a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.211150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c5032-492c-4302-966f-0218cb41531a service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.211433 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c5032-492c-4302-966f-0218cb41531a service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.211710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58c5032-492c-4302-966f-0218cb41531a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.213586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a58c5032-492c-4302-966f-0218cb41531a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.218204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 961/1923] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:23 2022] DELETE /identity/v3/users/7c2a9ed4780d486f8482876c4a78dda6 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:23.219724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726e002-e38d-4ffd-8759-06f16c6606fd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.220002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726e002-e38d-4ffd-8759-06f16c6606fd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.220284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9726e002-e38d-4ffd-8759-06f16c6606fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.229122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c5032-492c-4302-966f-0218cb41531a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.230636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58c5032-492c-4302-966f-0218cb41531a service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.239745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9726e002-e38d-4ffd-8759-06f16c6606fd None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:23.241311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 963/1924] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:23.241880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9726e002-e38d-4ffd-8759-06f16c6606fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.242788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9eea5-780e-4765-a6d5-0af8d2d4f5e9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.243074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9eea5-780e-4765-a6d5-0af8d2d4f5e9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.243482 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff9eea5-780e-4765-a6d5-0af8d2d4f5e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.266574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2ff9eea5-780e-4765-a6d5-0af8d2d4f5e9 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:23.267859 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2ff9eea5-780e-4765-a6d5-0af8d2d4f5e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.323844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 962/1925] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:23.328570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726e002-e38d-4ffd-8759-06f16c6606fd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.340991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9726e002-e38d-4ffd-8759-06f16c6606fd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.352298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b01d6-76f6-4cc6-af79-ca36900efe60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.352979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b01d6-76f6-4cc6-af79-ca36900efe60 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.353314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b01d6-76f6-4cc6-af79-ca36900efe60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.353879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b01d6-76f6-4cc6-af79-ca36900efe60 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.354281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b01d6-76f6-4cc6-af79-ca36900efe60 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:23.354594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b01d6-76f6-4cc6-af79-ca36900efe60 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.356206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488b01d6-76f6-4cc6-af79-ca36900efe60 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.374064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 964/1926] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:23.377353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 963/1927] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:23.378320 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9eea5-780e-4765-a6d5-0af8d2d4f5e9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.386463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b01d6-76f6-4cc6-af79-ca36900efe60 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.388805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-488b01d6-76f6-4cc6-af79-ca36900efe60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.390954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff9eea5-780e-4765-a6d5-0af8d2d4f5e9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.400421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc95e1-8c9c-4ade-a030-29bb212cb767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.401376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc95e1-8c9c-4ade-a030-29bb212cb767 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.401916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc95e1-8c9c-4ade-a030-29bb212cb767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.402281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44bc95e1-8c9c-4ade-a030-29bb212cb767 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.403101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc95e1-8c9c-4ade-a030-29bb212cb767 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.405778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44bc95e1-8c9c-4ade-a030-29bb212cb767 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.419256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:23.445724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 965/1928] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:23 2022] POST /identity/v3/projects => generated 385 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:23.446639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc95e1-8c9c-4ade-a030-29bb212cb767 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.475851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488b01d6-76f6-4cc6-af79-ca36900efe60 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.525814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:23.556105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44bc95e1-8c9c-4ade-a030-29bb212cb767 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.562649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539696-19ce-473b-a3e2-23e4c23f6f36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.563573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539696-19ce-473b-a3e2-23e4c23f6f36 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.564009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539696-19ce-473b-a3e2-23e4c23f6f36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.564426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44539696-19ce-473b-a3e2-23e4c23f6f36 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.565349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44539696-19ce-473b-a3e2-23e4c23f6f36 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.566513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c0457-6762-4312-8baf-e455600785ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.567173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0457-6762-4312-8baf-e455600785ae service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.567454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0457-6762-4312-8baf-e455600785ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.567728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7c0457-6762-4312-8baf-e455600785ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.568305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44539696-19ce-473b-a3e2-23e4c23f6f36 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.580481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c0457-6762-4312-8baf-e455600785ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.581999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7c0457-6762-4312-8baf-e455600785ae service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.594931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 964/1929] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:23.595937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 966/1930] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:23 2022] POST /identity/v3/users => generated 293 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:23.600401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c0457-6762-4312-8baf-e455600785ae service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.610579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7c0457-6762-4312-8baf-e455600785ae service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.615737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3095421-71dd-4e60-9a69-0fb45d04acbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.616418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3095421-71dd-4e60-9a69-0fb45d04acbd admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.616707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3095421-71dd-4e60-9a69-0fb45d04acbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.616991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3095421-71dd-4e60-9a69-0fb45d04acbd admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.617759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3095421-71dd-4e60-9a69-0fb45d04acbd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.619365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3095421-71dd-4e60-9a69-0fb45d04acbd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.633879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 965/1931] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:23.636093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539696-19ce-473b-a3e2-23e4c23f6f36 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.647205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44539696-19ce-473b-a3e2-23e4c23f6f36 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.652854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8c55f-330b-45ec-8979-051b6bc60e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.653565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8c55f-330b-45ec-8979-051b6bc60e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.653857 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8c55f-330b-45ec-8979-051b6bc60e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.654133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8c55f-330b-45ec-8979-051b6bc60e10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.654868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8c55f-330b-45ec-8979-051b6bc60e10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.656494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8c55f-330b-45ec-8979-051b6bc60e10 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.668889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 967/1932] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:23 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:23.674570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3095421-71dd-4e60-9a69-0fb45d04acbd admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.686572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3095421-71dd-4e60-9a69-0fb45d04acbd admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.691952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025a956-4359-4602-9d9d-ed213b175963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.693133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025a956-4359-4602-9d9d-ed213b175963 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.693889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025a956-4359-4602-9d9d-ed213b175963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.694525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f025a956-4359-4602-9d9d-ed213b175963 admin admin] PATH_INFO: `/v3/projects/7e1e312d370247289f51922a891919b0/users/7eb0a8c35ae641e382967dce8cf9fe65/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.731381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025a956-4359-4602-9d9d-ed213b175963 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e1e312d370247289f51922a891919b0, user_id=7eb0a8c35ae641e382967dce8cf9fe65, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.731381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f025a956-4359-4602-9d9d-ed213b175963 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:23.748671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 966/1933] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:23 2022] PUT /identity/v3/projects/7e1e312d370247289f51922a891919b0/users/7eb0a8c35ae641e382967dce8cf9fe65/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:23.762820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f41c-9b1d-4ca7-aff1-2f2fc1835cf4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.763327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f41c-9b1d-4ca7-aff1-2f2fc1835cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.763443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4270f41c-9b1d-4ca7-aff1-2f2fc1835cf4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.779825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025a956-4359-4602-9d9d-ed213b175963 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:23.782443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f025a956-4359-4602-9d9d-ed213b175963 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.844169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4270f41c-9b1d-4ca7-aff1-2f2fc1835cf4 None None] MFA Rules not processed for user `7eb0a8c35ae641e382967dce8cf9fe65`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:23.848200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4270f41c-9b1d-4ca7-aff1-2f2fc1835cf4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:23.970736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f025a956-4359-4602-9d9d-ed213b175963 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:23.986116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:23.989699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:23.991376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:23.991841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] PATH_INFO: `/v3/projects/d140ff225d104aa59e4262febe1a6b8e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:23.993819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d140ff225d104aa59e4262febe1a6b8e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:23.999620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:24.034581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 968/1934] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:23 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:24.038027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270f41c-9b1d-4ca7-aff1-2f2fc1835cf4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:24.063627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4270f41c-9b1d-4ca7-aff1-2f2fc1835cf4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:24.076076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110221cc-7597-443e-805b-27c46e41def0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:24.076787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110221cc-7597-443e-805b-27c46e41def0 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:24.077067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110221cc-7597-443e-805b-27c46e41def0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:24.077345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110221cc-7597-443e-805b-27c46e41def0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:24.079305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110221cc-7597-443e-805b-27c46e41def0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:24.086449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110221cc-7597-443e-805b-27c46e41def0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:24.118135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd140ff225d104aa59e4262febe1a6b8e', 'request_id': 'req-cd769872-8f99-4fcb-ba08-fd917610c654'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.133416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd140ff225d104aa59e4262febe1a6b8e', 'request_id': 'req-cd769872-8f99-4fcb-ba08-fd917610c654'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.148736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd140ff225d104aa59e4262febe1a6b8e', 'request_id': 'req-cd769872-8f99-4fcb-ba08-fd917610c654'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.151249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 969/1935] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:24 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:24.155753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110221cc-7597-443e-805b-27c46e41def0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:24.168824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] The token cache is being invalidate because project d140ff225d104aa59e4262febe1a6b8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:24.169427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.170521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.176128 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 967/1936] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:23 2022] DELETE /identity/v3/projects/d140ff225d104aa59e4262febe1a6b8e => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:24.177876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:24.181101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:24.216134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:24.275825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd769872-8f99-4fcb-ba08-fd917610c654 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:24.282396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:24.336346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110221cc-7597-443e-805b-27c46e41def0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:24.344726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0d627-3e54-4567-bd14-8b62b9fa99b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:24.345845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0d627-3e54-4567-bd14-8b62b9fa99b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:24.346316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0d627-3e54-4567-bd14-8b62b9fa99b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:24.346778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c0d627-3e54-4567-bd14-8b62b9fa99b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:24.347902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0d627-3e54-4567-bd14-8b62b9fa99b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:24.352766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c0d627-3e54-4567-bd14-8b62b9fa99b7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:24.405230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 970/1937] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:24 2022] POST /identity/v3/users => generated 299 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:24.406322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0d627-3e54-4567-bd14-8b62b9fa99b7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:24.421737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e4c0d627-3e54-4567-bd14-8b62b9fa99b7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:24.462329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c0d627-3e54-4567-bd14-8b62b9fa99b7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:24.477857 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:24.478610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:24.478738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:24.479075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:24.479558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] PATH_INFO: `/v3/projects/6d1b7b1c0cb149a0a0097de2bb286fde` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:24.479669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:24.479883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:24.480204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] PATH_INFO: `/v3/users/0ab7eb07874c413dbebe7e51ce042a1f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:24.480796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=6d1b7b1c0cb149a0a0097de2bb286fde)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:24.482208 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ab7eb07874c413dbebe7e51ce042a1f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:24.482827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:24.484660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:24.589890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6d1b7b1c0cb149a0a0097de2bb286fde', 'request_id': 'req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.598047 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6d1b7b1c0cb149a0a0097de2bb286fde', 'request_id': 'req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.602491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ab7eb07874c413dbebe7e51ce042a1f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.611114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6d1b7b1c0cb149a0a0097de2bb286fde', 'request_id': 'req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.629890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ab7eb07874c413dbebe7e51ce042a1f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.632239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] The token cache is being invalidate because project 6d1b7b1c0cb149a0a0097de2bb286fde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:24.632578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.633276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.634937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 971/1938] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:24 2022] DELETE /identity/v3/projects/6d1b7b1c0cb149a0a0097de2bb286fde => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:24.635717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:24.637311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:24.642410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ab7eb07874c413dbebe7e51ce042a1f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.643169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ab7eb07874c413dbebe7e51ce042a1f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.643836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ab7eb07874c413dbebe7e51ce042a1f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.669572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 968/1939] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:24 2022] DELETE /identity/v3/users/0ab7eb07874c413dbebe7e51ce042a1f => generated 0 bytes in 491 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:24.669572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:24.670271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:24.736896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc9265-7586-420c-bb89-7fa5b4cbcc3e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:24.750127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:24.774108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f608e43a-1310-4769-a54f-f1b899078985 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:24.782326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:24.783311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:24.783764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:24.784026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] PATH_INFO: `/v3/users/196d45883d23430695dd63b16c1a6693` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:24.785385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] RBAC: Authorizing `identity:delete_user(user_id=196d45883d23430695dd63b16c1a6693)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:24.788042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:24.809913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:24.828641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce1e26-5db9-4668-addf-6042ed16dd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:24.830130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce1e26-5db9-4668-addf-6042ed16dd85 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:24.832304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce1e26-5db9-4668-addf-6042ed16dd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:24.832304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fce1e26-5db9-4668-addf-6042ed16dd85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:24.832806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce1e26-5db9-4668-addf-6042ed16dd85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:24.837104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fce1e26-5db9-4668-addf-6042ed16dd85 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:24.857569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 972/1940] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:24.860579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce1e26-5db9-4668-addf-6042ed16dd85 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:24.862727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5fce1e26-5db9-4668-addf-6042ed16dd85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:24.870181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '196d45883d23430695dd63b16c1a6693'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.887001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fce1e26-5db9-4668-addf-6042ed16dd85 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:24.893243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3042994-ec11-4758-a312-503063468c53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:24.894480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '196d45883d23430695dd63b16c1a6693'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.894630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3042994-ec11-4758-a312-503063468c53 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:24.894881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3042994-ec11-4758-a312-503063468c53 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:24.895162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3042994-ec11-4758-a312-503063468c53 admin admin] PATH_INFO: `/v3/projects/2963c83f31cf4753b33bee82edc8716a/users/43a96ed742e94e248ddf77d57cec1915/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:24.904059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '196d45883d23430695dd63b16c1a6693'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.905077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '196d45883d23430695dd63b16c1a6693'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.905870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '196d45883d23430695dd63b16c1a6693'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:24.918377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3042994-ec11-4758-a312-503063468c53 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2963c83f31cf4753b33bee82edc8716a, user_id=43a96ed742e94e248ddf77d57cec1915, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:24.921117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3042994-ec11-4758-a312-503063468c53 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:24.928034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 969/1941] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:24 2022] DELETE /identity/v3/users/196d45883d23430695dd63b16c1a6693 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:24.942898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 973/1942] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:24 2022] PUT /identity/v3/projects/2963c83f31cf4753b33bee82edc8716a/users/43a96ed742e94e248ddf77d57cec1915/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:24.947860 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:24.949554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.023374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc85c3f-7c84-4785-a982-881b9be5e31a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:25.075537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3042994-ec11-4758-a312-503063468c53 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:25.078135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b3042994-ec11-4758-a312-503063468c53 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.100015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab5786-6904-4934-9bf2-a0c817c791bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:25.100726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab5786-6904-4934-9bf2-a0c817c791bc admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.101220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab5786-6904-4934-9bf2-a0c817c791bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.101798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ab5786-6904-4934-9bf2-a0c817c791bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.102665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab5786-6904-4934-9bf2-a0c817c791bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:25.104202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ab5786-6904-4934-9bf2-a0c817c791bc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:25.123066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 970/1943] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:24 2022] GET /identity/v3/roles => generated 1420 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:25.125822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab5786-6904-4934-9bf2-a0c817c791bc admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:25.142912 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ab5786-6904-4934-9bf2-a0c817c791bc admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:25.150579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bc68d3-f5c6-41b5-a523-f1e253813fd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:25.151528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc68d3-f5c6-41b5-a523-f1e253813fd2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.151764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc68d3-f5c6-41b5-a523-f1e253813fd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.152188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bc68d3-f5c6-41b5-a523-f1e253813fd2 admin admin] PATH_INFO: `/v3/projects/2963c83f31cf4753b33bee82edc8716a/users/43a96ed742e94e248ddf77d57cec1915/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.154803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc68d3-f5c6-41b5-a523-f1e253813fd2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2963c83f31cf4753b33bee82edc8716a, user_id=43a96ed742e94e248ddf77d57cec1915, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:25.159033 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bc68d3-f5c6-41b5-a523-f1e253813fd2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:25.166018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3042994-ec11-4758-a312-503063468c53 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:25.167109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:25.175985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 971/1944] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:25 2022] PUT /identity/v3/projects/2963c83f31cf4753b33bee82edc8716a/users/43a96ed742e94e248ddf77d57cec1915/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:25.187050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb90b19-3e91-4e14-bfac-8563662250c0 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.187319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb90b19-3e91-4e14-bfac-8563662250c0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.187574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb90b19-3e91-4e14-bfac-8563662250c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.239181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8cb90b19-3e91-4e14-bfac-8563662250c0 None None] MFA Rules not processed for user `43a96ed742e94e248ddf77d57cec1915`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:25.241066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8cb90b19-3e91-4e14-bfac-8563662250c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.267913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:25.268815 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.269104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.269386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] PATH_INFO: `/v3/projects/71eba902a494470cad8a0cc764e344e8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.272016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] RBAC: Authorizing `identity:delete_project(project_id=71eba902a494470cad8a0cc764e344e8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:25.274058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:25.384100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '71eba902a494470cad8a0cc764e344e8', 'request_id': 'req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:25.394889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '71eba902a494470cad8a0cc764e344e8', 'request_id': 'req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:25.400616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 972/1945] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:25.401307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb90b19-3e91-4e14-bfac-8563662250c0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:25.402660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8cb90b19-3e91-4e14-bfac-8563662250c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.414197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '71eba902a494470cad8a0cc764e344e8', 'request_id': 'req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:25.422213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb90b19-3e91-4e14-bfac-8563662250c0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:25.430878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] The token cache is being invalidate because project 71eba902a494470cad8a0cc764e344e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:25.431202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:25.432540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:25.434198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 974/1946] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:25 2022] DELETE /identity/v3/projects/71eba902a494470cad8a0cc764e344e8 => generated 0 bytes in 359 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:25.439254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:25.441302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.472522 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:25.524259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:25.525244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.525552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.526003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] PATH_INFO: `/v3/projects/689a18fa26f04804b4633df4b2bda599` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.527346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] RBAC: Authorizing `identity:delete_project(project_id=689a18fa26f04804b4633df4b2bda599)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:25.529543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:25.536570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4ba359-d7e8-4f97-b5bb-c70aa033b167 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:25.541828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd92492e-887b-4a7b-a98f-8254d9ff719c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:25.543278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92492e-887b-4a7b-a98f-8254d9ff719c service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.543701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92492e-887b-4a7b-a98f-8254d9ff719c service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.543992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd92492e-887b-4a7b-a98f-8254d9ff719c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.550345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-dd92492e-887b-4a7b-a98f-8254d9ff719c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.614737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '689a18fa26f04804b4633df4b2bda599', 'request_id': 'req-15340732-6cbc-4b05-9a59-89055ffc6187'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:25.623566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '689a18fa26f04804b4633df4b2bda599', 'request_id': 'req-15340732-6cbc-4b05-9a59-89055ffc6187'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:25.624924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92492e-887b-4a7b-a98f-8254d9ff719c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:25.626690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd92492e-887b-4a7b-a98f-8254d9ff719c service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:25.637544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '689a18fa26f04804b4633df4b2bda599', 'request_id': 'req-15340732-6cbc-4b05-9a59-89055ffc6187'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:25.641679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 975/1947] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:25 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:25.643212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cfb63-4e33-4c3a-a020-686970f074cf None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.643515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cfb63-4e33-4c3a-a020-686970f074cf None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.643768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cfb63-4e33-4c3a-a020-686970f074cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.657011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] The token cache is being invalidate because project 689a18fa26f04804b4633df4b2bda599 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:25.657374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:25.658453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-15340732-6cbc-4b05-9a59-89055ffc6187 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:25.665755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 973/1948] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:25 2022] DELETE /identity/v3/projects/689a18fa26f04804b4633df4b2bda599 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:25.667152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfe18d-d4f4-4063-9d34-f9163ae608f3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.667389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfe18d-d4f4-4063-9d34-f9163ae608f3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.667640 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21bfe18d-d4f4-4063-9d34-f9163ae608f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.675169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-371cfb63-4e33-4c3a-a020-686970f074cf None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:25.676441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-371cfb63-4e33-4c3a-a020-686970f074cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.700789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-21bfe18d-d4f4-4063-9d34-f9163ae608f3 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:25.703722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-21bfe18d-d4f4-4063-9d34-f9163ae608f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.717877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:25.781637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 976/1949] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:25.790354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c24a20-03fd-4d68-aeef-d35a40fa2661 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.790677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c24a20-03fd-4d68-aeef-d35a40fa2661 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.791036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c24a20-03fd-4d68-aeef-d35a40fa2661 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.815085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 974/1950] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:25.832444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87625eb9-36f0-4bc7-99cd-0367ab714579 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.832911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87625eb9-36f0-4bc7-99cd-0367ab714579 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.833321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87625eb9-36f0-4bc7-99cd-0367ab714579 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.849447 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f8c24a20-03fd-4d68-aeef-d35a40fa2661 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:25.851681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f8c24a20-03fd-4d68-aeef-d35a40fa2661 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.863069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 977/1951] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:25.868565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c24a20-03fd-4d68-aeef-d35a40fa2661 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:25.875755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-87625eb9-36f0-4bc7-99cd-0367ab714579 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:25.877968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-87625eb9-36f0-4bc7-99cd-0367ab714579 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:25.882788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c24a20-03fd-4d68-aeef-d35a40fa2661 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:25.887795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 975/1952] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:25.895713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87625eb9-36f0-4bc7-99cd-0367ab714579 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:25.899572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebb1f6-4760-443b-bab8-5c92ad257cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:25.900900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebb1f6-4760-443b-bab8-5c92ad257cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.901745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebb1f6-4760-443b-bab8-5c92ad257cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.902304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aebb1f6-4760-443b-bab8-5c92ad257cdc admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.903735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebb1f6-4760-443b-bab8-5c92ad257cdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:25.904355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebb1f6-4760-443b-bab8-5c92ad257cdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:25.908698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aebb1f6-4760-443b-bab8-5c92ad257cdc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:25.917359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87625eb9-36f0-4bc7-99cd-0367ab714579 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:25.932875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 978/1953] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:25.933842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebb1f6-4760-443b-bab8-5c92ad257cdc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:25.938598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254709ae-0de8-48e9-9236-97ca84dbf38a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:25.939314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254709ae-0de8-48e9-9236-97ca84dbf38a admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.939589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254709ae-0de8-48e9-9236-97ca84dbf38a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.940563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254709ae-0de8-48e9-9236-97ca84dbf38a admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.941311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254709ae-0de8-48e9-9236-97ca84dbf38a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:25.941660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254709ae-0de8-48e9-9236-97ca84dbf38a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:25.945169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254709ae-0de8-48e9-9236-97ca84dbf38a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:25.955029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aebb1f6-4760-443b-bab8-5c92ad257cdc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:25.963247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67454f3e-9432-4a29-802d-ee7a11a1f37c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:25.964513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67454f3e-9432-4a29-802d-ee7a11a1f37c admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:25.966519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67454f3e-9432-4a29-802d-ee7a11a1f37c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:25.967334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67454f3e-9432-4a29-802d-ee7a11a1f37c admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:25.970140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67454f3e-9432-4a29-802d-ee7a11a1f37c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:25.974105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67454f3e-9432-4a29-802d-ee7a11a1f37c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:25.979879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 976/1954] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:24:25.980728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254709ae-0de8-48e9-9236-97ca84dbf38a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:25.994452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254709ae-0de8-48e9-9236-97ca84dbf38a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.008042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5b921-bb0e-4990-8353-304564a716fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.008840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5b921-bb0e-4990-8353-304564a716fc admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.009385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5b921-bb0e-4990-8353-304564a716fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.009705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb5b921-bb0e-4990-8353-304564a716fc admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.010411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5b921-bb0e-4990-8353-304564a716fc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.012743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb5b921-bb0e-4990-8353-304564a716fc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.024747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 979/1955] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:25 2022] POST /identity/v3/projects => generated 373 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:26.025776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67454f3e-9432-4a29-802d-ee7a11a1f37c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.057559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 977/1956] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:25 2022] POST /identity/v3/projects => generated 377 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:26.061526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5b921-bb0e-4990-8353-304564a716fc admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.079323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:26.180420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67454f3e-9432-4a29-802d-ee7a11a1f37c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.183556 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb5b921-bb0e-4990-8353-304564a716fc admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.185791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dcb45-6c98-4adb-ac9a-5c0e1398cd13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.190270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c88ba-8ded-481f-ad07-a0e185f89358 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.191230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c88ba-8ded-481f-ad07-a0e185f89358 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.191623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c88ba-8ded-481f-ad07-a0e185f89358 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.192006 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c88ba-8ded-481f-ad07-a0e185f89358 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.192884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c88ba-8ded-481f-ad07-a0e185f89358 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.194299 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dcb45-6c98-4adb-ac9a-5c0e1398cd13 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.194711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dcb45-6c98-4adb-ac9a-5c0e1398cd13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.195004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c0dcb45-6c98-4adb-ac9a-5c0e1398cd13 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.195621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc8c88ba-8ded-481f-ad07-a0e185f89358 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.195733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dcb45-6c98-4adb-ac9a-5c0e1398cd13 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.201380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c0dcb45-6c98-4adb-ac9a-5c0e1398cd13 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.233292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 978/1957] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:26 2022] POST /identity/v3/users => generated 289 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:26.235714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 980/1958] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:26 2022] POST /identity/v3/users => generated 287 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:26.237281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c88ba-8ded-481f-ad07-a0e185f89358 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.240591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dcb45-6c98-4adb-ac9a-5c0e1398cd13 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.249547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8c88ba-8ded-481f-ad07-a0e185f89358 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.252985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c0dcb45-6c98-4adb-ac9a-5c0e1398cd13 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.255524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f441461-5cf9-4e52-8926-9b7264f28e87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.256275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f441461-5cf9-4e52-8926-9b7264f28e87 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.256564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f441461-5cf9-4e52-8926-9b7264f28e87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.256851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f441461-5cf9-4e52-8926-9b7264f28e87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.257843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f441461-5cf9-4e52-8926-9b7264f28e87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.259808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f441461-5cf9-4e52-8926-9b7264f28e87 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.264026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ae2af-9cc3-44ef-8cea-1a3080fc7eef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.265565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ae2af-9cc3-44ef-8cea-1a3080fc7eef admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.266250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ae2af-9cc3-44ef-8cea-1a3080fc7eef admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.267063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8ae2af-9cc3-44ef-8cea-1a3080fc7eef admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.268355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ae2af-9cc3-44ef-8cea-1a3080fc7eef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.272376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8ae2af-9cc3-44ef-8cea-1a3080fc7eef admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.273435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 979/1959] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:26.278622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f441461-5cf9-4e52-8926-9b7264f28e87 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.290621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f441461-5cf9-4e52-8926-9b7264f28e87 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.303404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 981/1960] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:26.303583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c16f4d-e39b-4f5f-a62f-dbdab4a61d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.304093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c16f4d-e39b-4f5f-a62f-dbdab4a61d3b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.304386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c16f4d-e39b-4f5f-a62f-dbdab4a61d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.304669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c16f4d-e39b-4f5f-a62f-dbdab4a61d3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.305734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c16f4d-e39b-4f5f-a62f-dbdab4a61d3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.305920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ae2af-9cc3-44ef-8cea-1a3080fc7eef admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.307532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c16f4d-e39b-4f5f-a62f-dbdab4a61d3b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.324414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8ae2af-9cc3-44ef-8cea-1a3080fc7eef admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.325097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 980/1961] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:26.330989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c16f4d-e39b-4f5f-a62f-dbdab4a61d3b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.342744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e190431-58e0-4209-be11-f51e45d2235a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.343720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e190431-58e0-4209-be11-f51e45d2235a admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.344168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e190431-58e0-4209-be11-f51e45d2235a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.344547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e190431-58e0-4209-be11-f51e45d2235a admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.344850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c16f4d-e39b-4f5f-a62f-dbdab4a61d3b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.346972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e190431-58e0-4209-be11-f51e45d2235a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.351196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009dd89f-b017-4f1e-a7f3-204455def5fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.351379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e190431-58e0-4209-be11-f51e45d2235a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.352109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dd89f-b017-4f1e-a7f3-204455def5fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.352395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dd89f-b017-4f1e-a7f3-204455def5fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.352816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009dd89f-b017-4f1e-a7f3-204455def5fb admin admin] PATH_INFO: `/v3/projects/98d8e283b73f48369a18c53bdb965876/users/eb7ab10961cf46409877514c54e23dbc/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.369995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 982/1962] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:26.377986 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e190431-58e0-4209-be11-f51e45d2235a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.385011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009dd89f-b017-4f1e-a7f3-204455def5fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=98d8e283b73f48369a18c53bdb965876, user_id=eb7ab10961cf46409877514c54e23dbc, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.386782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e190431-58e0-4209-be11-f51e45d2235a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.388841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009dd89f-b017-4f1e-a7f3-204455def5fb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.391575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1988a45c-91e3-42b9-9583-fa18f0e2f6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.392460 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988a45c-91e3-42b9-9583-fa18f0e2f6a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.392745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988a45c-91e3-42b9-9583-fa18f0e2f6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.393556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1988a45c-91e3-42b9-9583-fa18f0e2f6a4 admin admin] PATH_INFO: `/v3/projects/a51cf9bd2ea44ebe8edd25559baafc4e/users/afa79c7408524e979cf0e9a595fe731a/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.410147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 981/1963] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:26 2022] PUT /identity/v3/projects/98d8e283b73f48369a18c53bdb965876/users/eb7ab10961cf46409877514c54e23dbc/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:26.416389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988a45c-91e3-42b9-9583-fa18f0e2f6a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a51cf9bd2ea44ebe8edd25559baafc4e, user_id=afa79c7408524e979cf0e9a595fe731a, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.420500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1988a45c-91e3-42b9-9583-fa18f0e2f6a4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.422041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a587da-13ff-481e-8c9b-6b575b872934 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.422690 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a587da-13ff-481e-8c9b-6b575b872934 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.424338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a587da-13ff-481e-8c9b-6b575b872934 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.441623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 983/1964] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:26 2022] PUT /identity/v3/projects/a51cf9bd2ea44ebe8edd25559baafc4e/users/afa79c7408524e979cf0e9a595fe731a/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:26.449175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af079390-f4e1-48f6-9c2f-c89599d13c50 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.449581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af079390-f4e1-48f6-9c2f-c89599d13c50 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.449995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af079390-f4e1-48f6-9c2f-c89599d13c50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.463757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-25a587da-13ff-481e-8c9b-6b575b872934 None None] MFA Rules not processed for user `eb7ab10961cf46409877514c54e23dbc`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:26.465484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-25a587da-13ff-481e-8c9b-6b575b872934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:26.470369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-af079390-f4e1-48f6-9c2f-c89599d13c50 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:26.471299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:26.473667 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-af079390-f4e1-48f6-9c2f-c89599d13c50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:26.551860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 984/1965] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:26.558498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 982/1966] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:26.559767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af079390-f4e1-48f6-9c2f-c89599d13c50 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.560448 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a587da-13ff-481e-8c9b-6b575b872934 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.562151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-25a587da-13ff-481e-8c9b-6b575b872934 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:26.571088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af079390-f4e1-48f6-9c2f-c89599d13c50 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.578194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07347-0f0c-4240-959c-ed94eb9e83f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.579486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07347-0f0c-4240-959c-ed94eb9e83f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.579983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07347-0f0c-4240-959c-ed94eb9e83f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.580485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d07347-0f0c-4240-959c-ed94eb9e83f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.581457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07347-0f0c-4240-959c-ed94eb9e83f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.585049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d07347-0f0c-4240-959c-ed94eb9e83f7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.624394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 985/1967] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:26 2022] POST /identity/v3/projects => generated 379 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:26.628296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07347-0f0c-4240-959c-ed94eb9e83f7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.645756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a587da-13ff-481e-8c9b-6b575b872934 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.709724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:26.745710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d07347-0f0c-4240-959c-ed94eb9e83f7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.763564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433559ca-37cb-460e-9c2b-7be5243aefbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.764452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433559ca-37cb-460e-9c2b-7be5243aefbb admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.764804 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433559ca-37cb-460e-9c2b-7be5243aefbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.765156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433559ca-37cb-460e-9c2b-7be5243aefbb admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.765956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433559ca-37cb-460e-9c2b-7be5243aefbb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.766624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57c6c7-14c3-4461-84c4-844f4625ba76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.776192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57c6c7-14c3-4461-84c4-844f4625ba76 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.776192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57c6c7-14c3-4461-84c4-844f4625ba76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.776192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a57c6c7-14c3-4461-84c4-844f4625ba76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.776808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433559ca-37cb-460e-9c2b-7be5243aefbb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.780231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57c6c7-14c3-4461-84c4-844f4625ba76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.781897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a57c6c7-14c3-4461-84c4-844f4625ba76 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.797371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 983/1968] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:26.801830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 986/1969] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:26 2022] POST /identity/v3/users => generated 290 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:26.803664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57c6c7-14c3-4461-84c4-844f4625ba76 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.814334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a57c6c7-14c3-4461-84c4-844f4625ba76 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.820260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9e1c-b54b-4857-9519-ef0d3a6b21a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.821095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9e1c-b54b-4857-9519-ef0d3a6b21a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.821388 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9e1c-b54b-4857-9519-ef0d3a6b21a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.821710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c9e1c-b54b-4857-9519-ef0d3a6b21a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.822473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c9e1c-b54b-4857-9519-ef0d3a6b21a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.824091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741c9e1c-b54b-4857-9519-ef0d3a6b21a2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.837184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 984/1970] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:26.841177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433559ca-37cb-460e-9c2b-7be5243aefbb admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.853050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433559ca-37cb-460e-9c2b-7be5243aefbb admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.858528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea860a71-f89c-44ce-a3d8-8d274bb361c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.859140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea860a71-f89c-44ce-a3d8-8d274bb361c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.859427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea860a71-f89c-44ce-a3d8-8d274bb361c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.859717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea860a71-f89c-44ce-a3d8-8d274bb361c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.860640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea860a71-f89c-44ce-a3d8-8d274bb361c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.862305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea860a71-f89c-44ce-a3d8-8d274bb361c4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.874217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 987/1971] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:26 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:26.878370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9e1c-b54b-4857-9519-ef0d3a6b21a2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:26.889722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741c9e1c-b54b-4857-9519-ef0d3a6b21a2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:26.894671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861351a-2f49-4bad-b42c-cb9b7eeb8271 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:26.895577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861351a-2f49-4bad-b42c-cb9b7eeb8271 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.895849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861351a-2f49-4bad-b42c-cb9b7eeb8271 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.896138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8861351a-2f49-4bad-b42c-cb9b7eeb8271 admin admin] PATH_INFO: `/v3/projects/2bbd936c1ec64798ad6557a5376cc32c/users/8fea3d4a0000488696f268d57a570b07/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.914753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861351a-2f49-4bad-b42c-cb9b7eeb8271 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2bbd936c1ec64798ad6557a5376cc32c, user_id=8fea3d4a0000488696f268d57a570b07, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:26.917600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8861351a-2f49-4bad-b42c-cb9b7eeb8271 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:26.949452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313a029-e052-4eb0-8b49-40e792c66db1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:26.949822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313a029-e052-4eb0-8b49-40e792c66db1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:26.950069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5313a029-e052-4eb0-8b49-40e792c66db1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:26.960908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 985/1972] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:26 2022] PUT /identity/v3/projects/2bbd936c1ec64798ad6557a5376cc32c/users/8fea3d4a0000488696f268d57a570b07/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:27.000288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5313a029-e052-4eb0-8b49-40e792c66db1 None None] MFA Rules not processed for user `8fea3d4a0000488696f268d57a570b07`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:27.001754 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5313a029-e052-4eb0-8b49-40e792c66db1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:27.084792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 988/1973] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:26 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:27.089282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861351a-2f49-4bad-b42c-cb9b7eeb8271 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:27.138508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:27.209437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8861351a-2f49-4bad-b42c-cb9b7eeb8271 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:27.222128 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fafa0-59fe-4c81-96dd-436b6b2266a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:27.222921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fafa0-59fe-4c81-96dd-436b6b2266a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:27.223217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fafa0-59fe-4c81-96dd-436b6b2266a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:27.223533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-161fafa0-59fe-4c81-96dd-436b6b2266a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:27.224315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fafa0-59fe-4c81-96dd-436b6b2266a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:27.227969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-161fafa0-59fe-4c81-96dd-436b6b2266a7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:27.265887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 986/1974] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:27 2022] POST /identity/v3/projects => generated 379 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:27.269806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313a029-e052-4eb0-8b49-40e792c66db1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:27.284257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5313a029-e052-4eb0-8b49-40e792c66db1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:27.390105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8e46e-535c-4f2d-b2f1-61a87a6ac3b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:27.391093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8e46e-535c-4f2d-b2f1-61a87a6ac3b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:27.391488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8e46e-535c-4f2d-b2f1-61a87a6ac3b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:27.391947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d8e46e-535c-4f2d-b2f1-61a87a6ac3b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:27.393398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8e46e-535c-4f2d-b2f1-61a87a6ac3b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:27.395698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d8e46e-535c-4f2d-b2f1-61a87a6ac3b2 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:27.435402 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 989/1975] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:27 2022] POST /identity/v3/users => generated 296 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:27.436779 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fafa0-59fe-4c81-96dd-436b6b2266a7 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:27.455227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-161fafa0-59fe-4c81-96dd-436b6b2266a7 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:27.461950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664cd8c-d3e4-4adf-8bd4-28e1bba647c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:27.462896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664cd8c-d3e4-4adf-8bd4-28e1bba647c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:27.464351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664cd8c-d3e4-4adf-8bd4-28e1bba647c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:27.464674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4664cd8c-d3e4-4adf-8bd4-28e1bba647c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:27.465605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664cd8c-d3e4-4adf-8bd4-28e1bba647c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:27.467573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4664cd8c-d3e4-4adf-8bd4-28e1bba647c1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:27.487054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 987/1976] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:27.491890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8e46e-535c-4f2d-b2f1-61a87a6ac3b2 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:27.507656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d8e46e-535c-4f2d-b2f1-61a87a6ac3b2 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:27.522421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38e1f2-a6c5-4a99-87d5-7d3cc365596d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:27.526088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38e1f2-a6c5-4a99-87d5-7d3cc365596d admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:27.526489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38e1f2-a6c5-4a99-87d5-7d3cc365596d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:27.527170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea38e1f2-a6c5-4a99-87d5-7d3cc365596d admin admin] PATH_INFO: `/v3/projects/2a1a14d3054e4e6e82c014ca047fd0b3/users/ec74792aab614cf98073e8202a64088f/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:27.579378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38e1f2-a6c5-4a99-87d5-7d3cc365596d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1a14d3054e4e6e82c014ca047fd0b3, user_id=ec74792aab614cf98073e8202a64088f, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:27.584428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea38e1f2-a6c5-4a99-87d5-7d3cc365596d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:27.614094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 990/1977] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:27 2022] PUT /identity/v3/projects/2a1a14d3054e4e6e82c014ca047fd0b3/users/ec74792aab614cf98073e8202a64088f/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:27.615638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664cd8c-d3e4-4adf-8bd4-28e1bba647c1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:27.633947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4664cd8c-d3e4-4adf-8bd4-28e1bba647c1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:27.780711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3fed-1322-486a-8a5f-c8dfb280711c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:27.781408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3fed-1322-486a-8a5f-c8dfb280711c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:27.781725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3fed-1322-486a-8a5f-c8dfb280711c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:27.782087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3fed-1322-486a-8a5f-c8dfb280711c admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:27.782749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f3fed-1322-486a-8a5f-c8dfb280711c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:27.784406 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f3fed-1322-486a-8a5f-c8dfb280711c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:27.803399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 988/1978] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:27 2022] GET /identity/v3/roles => generated 1420 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:27.804547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38e1f2-a6c5-4a99-87d5-7d3cc365596d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:27.821101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea38e1f2-a6c5-4a99-87d5-7d3cc365596d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:27.859539 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7e968-b258-4d0f-8023-c849ddb8ff5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:27.865037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7e968-b258-4d0f-8023-c849ddb8ff5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:27.865500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7e968-b258-4d0f-8023-c849ddb8ff5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:27.867129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b7e968-b258-4d0f-8023-c849ddb8ff5b admin admin] PATH_INFO: `/v3/projects/2a1a14d3054e4e6e82c014ca047fd0b3/users/ec74792aab614cf98073e8202a64088f/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:27.875883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7e968-b258-4d0f-8023-c849ddb8ff5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a1a14d3054e4e6e82c014ca047fd0b3, user_id=ec74792aab614cf98073e8202a64088f, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:27.880531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03b7e968-b258-4d0f-8023-c849ddb8ff5b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:27.927886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965201c-2220-46f9-91ca-e7b451c7d6e4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:27.928376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965201c-2220-46f9-91ca-e7b451c7d6e4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:27.928853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9965201c-2220-46f9-91ca-e7b451c7d6e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:27.930700 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 991/1979] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:27 2022] PUT /identity/v3/projects/2a1a14d3054e4e6e82c014ca047fd0b3/users/ec74792aab614cf98073e8202a64088f/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:27.969235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-9965201c-2220-46f9-91ca-e7b451c7d6e4 None None] MFA Rules not processed for user `ec74792aab614cf98073e8202a64088f`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:27.970893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9965201c-2220-46f9-91ca-e7b451c7d6e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:28.074433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 989/1980] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:28.087688 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7e968-b258-4d0f-8023-c849ddb8ff5b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:28.089494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-03b7e968-b258-4d0f-8023-c849ddb8ff5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:28.096377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:28.177927 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b7e968-b258-4d0f-8023-c849ddb8ff5b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:28.183157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b936f8f-416a-4212-a9ba-6830ff85b87c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:28.184180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b936f8f-416a-4212-a9ba-6830ff85b87c service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:28.184562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b936f8f-416a-4212-a9ba-6830ff85b87c service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:28.185018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b936f8f-416a-4212-a9ba-6830ff85b87c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:28.202287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b936f8f-416a-4212-a9ba-6830ff85b87c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:28.204703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b936f8f-416a-4212-a9ba-6830ff85b87c service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:28.238106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 992/1981] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:28 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:28.262160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20674d-1f44-4ac0-b618-9e7f728ed7b1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:28.262733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20674d-1f44-4ac0-b618-9e7f728ed7b1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:28.265356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20674d-1f44-4ac0-b618-9e7f728ed7b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:28.316238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8d20674d-1f44-4ac0-b618-9e7f728ed7b1 None None] MFA Rules not processed for user `afa79c7408524e979cf0e9a595fe731a`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:28.316238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8d20674d-1f44-4ac0-b618-9e7f728ed7b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:28.326622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b936f8f-416a-4212-a9ba-6830ff85b87c service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:28.328283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5b936f8f-416a-4212-a9ba-6830ff85b87c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:28.333503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:28.347908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b936f8f-416a-4212-a9ba-6830ff85b87c service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:28.352747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957931f6-3200-4f4b-959c-a76fe57ad11d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:28.353420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957931f6-3200-4f4b-959c-a76fe57ad11d service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:28.353741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957931f6-3200-4f4b-959c-a76fe57ad11d service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:28.354023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957931f6-3200-4f4b-959c-a76fe57ad11d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:28.435132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 990/1982] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:28.453158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20674d-1f44-4ac0-b618-9e7f728ed7b1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:28.466333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20674d-1f44-4ac0-b618-9e7f728ed7b1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:28.470940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68437a50-fb37-49ab-bc20-03f8023db1cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:28.471613 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68437a50-fb37-49ab-bc20-03f8023db1cf service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:28.472009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68437a50-fb37-49ab-bc20-03f8023db1cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:28.472295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68437a50-fb37-49ab-bc20-03f8023db1cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:28.485851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68437a50-fb37-49ab-bc20-03f8023db1cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:28.487336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68437a50-fb37-49ab-bc20-03f8023db1cf service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:28.490558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957931f6-3200-4f4b-959c-a76fe57ad11d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:28.502547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957931f6-3200-4f4b-959c-a76fe57ad11d service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:28.504029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 991/1983] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:28.514027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 993/1984] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:28 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:31.101009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3d28b-366b-49b8-ae18-d6b0bdcbc903 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:31.101793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3d28b-366b-49b8-ae18-d6b0bdcbc903 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:31.101793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b3d28b-366b-49b8-ae18-d6b0bdcbc903 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:31.140513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f1b3d28b-366b-49b8-ae18-d6b0bdcbc903 None None] MFA Rules not processed for user `afa79c7408524e979cf0e9a595fe731a`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:31.141799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f1b3d28b-366b-49b8-ae18-d6b0bdcbc903 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:31.151758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 992/1985] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:31.649965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957931f6-3200-4f4b-959c-a76fe57ad11d service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:31.661044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957931f6-3200-4f4b-959c-a76fe57ad11d service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:31.668718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92756e2-31b1-4956-9077-ed2c4c6bd30e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:31.669536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92756e2-31b1-4956-9077-ed2c4c6bd30e service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:31.669981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92756e2-31b1-4956-9077-ed2c4c6bd30e service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:31.670352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92756e2-31b1-4956-9077-ed2c4c6bd30e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:31.672277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d92756e2-31b1-4956-9077-ed2c4c6bd30e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:31.743385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92756e2-31b1-4956-9077-ed2c4c6bd30e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:31.744856 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92756e2-31b1-4956-9077-ed2c4c6bd30e service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:31.755707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 994/1986] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:31 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:34.902609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3d28b-366b-49b8-ae18-d6b0bdcbc903 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:34.983613 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b3d28b-366b-49b8-ae18-d6b0bdcbc903 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:34.991894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:34.993458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:34.994016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:34.994526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] PATH_INFO: `/v3/users/eb7ab10961cf46409877514c54e23dbc` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:34.996360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb7ab10961cf46409877514c54e23dbc)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:34.999476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:35.056018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb7ab10961cf46409877514c54e23dbc'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:35.076180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb7ab10961cf46409877514c54e23dbc'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:35.085811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb7ab10961cf46409877514c54e23dbc'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:35.086532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb7ab10961cf46409877514c54e23dbc'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:35.087166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb7ab10961cf46409877514c54e23dbc'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:35.104294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 993/1987] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:34 2022] DELETE /identity/v3/users/eb7ab10961cf46409877514c54e23dbc => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:35.108944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d28b0c-3d23-4cf0-8528-ae4a4c34c064 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:35.109457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d28b0c-3d23-4cf0-8528-ae4a4c34c064 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:35.109587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d28b0c-3d23-4cf0-8528-ae4a4c34c064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:35.128135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-12d28b0c-3d23-4cf0-8528-ae4a4c34c064 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:35.129354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-12d28b0c-3d23-4cf0-8528-ae4a4c34c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:35.176920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:35.214046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 995/1988] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:35.222114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:35.224891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:35.244262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:35.297734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52008dd5-a58c-4056-81af-8a24dd72be04 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:35.303702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baba31-2637-4f66-9e60-b5094e2204ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:35.304379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baba31-2637-4f66-9e60-b5094e2204ef service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:35.304661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baba31-2637-4f66-9e60-b5094e2204ef service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:35.304939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69baba31-2637-4f66-9e60-b5094e2204ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:35.315740 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69baba31-2637-4f66-9e60-b5094e2204ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:35.317577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69baba31-2637-4f66-9e60-b5094e2204ef service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:35.330881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 994/1989] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:35.775759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d28b0c-3d23-4cf0-8528-ae4a4c34c064 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:35.777596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-12d28b0c-3d23-4cf0-8528-ae4a4c34c064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:35.797077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d28b0c-3d23-4cf0-8528-ae4a4c34c064 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:35.805894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:35.807105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:35.807567 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:35.808018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] PATH_INFO: `/v3/projects/98d8e283b73f48369a18c53bdb965876` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:35.810081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] RBAC: Authorizing `identity:delete_project(project_id=98d8e283b73f48369a18c53bdb965876)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:35.813076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:35.870197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '98d8e283b73f48369a18c53bdb965876', 'request_id': 'req-8541c9ba-1d29-4230-a6c1-73e7677eb782'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:35.876547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '98d8e283b73f48369a18c53bdb965876', 'request_id': 'req-8541c9ba-1d29-4230-a6c1-73e7677eb782'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:35.885020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '98d8e283b73f48369a18c53bdb965876', 'request_id': 'req-8541c9ba-1d29-4230-a6c1-73e7677eb782'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:35.898800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] The token cache is being invalidate because project 98d8e283b73f48369a18c53bdb965876 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:35.899092 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:35.899772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:35.901635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 996/1990] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:35 2022] DELETE /identity/v3/projects/98d8e283b73f48369a18c53bdb965876 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:35.905634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baba31-2637-4f66-9e60-b5094e2204ef service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:35.917120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69baba31-2637-4f66-9e60-b5094e2204ef service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:35.998138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:35.999153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:35.999501 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:35.999842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] PATH_INFO: `/v3/users/8fea3d4a0000488696f268d57a570b07` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:36.001384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fea3d4a0000488696f268d57a570b07)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:36.003791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:36.077900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fea3d4a0000488696f268d57a570b07'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.096116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fea3d4a0000488696f268d57a570b07'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.106217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fea3d4a0000488696f268d57a570b07'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.107032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fea3d4a0000488696f268d57a570b07'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.108061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fea3d4a0000488696f268d57a570b07'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.124883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 995/1991] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:35 2022] DELETE /identity/v3/users/8fea3d4a0000488696f268d57a570b07 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:36.545383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:36.546938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:36.560564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8541c9ba-1d29-4230-a6c1-73e7677eb782 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:36.625401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:36.626236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:36.626520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:36.627299 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] PATH_INFO: `/v3/projects/2bbd936c1ec64798ad6557a5376cc32c` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:36.628516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2bbd936c1ec64798ad6557a5376cc32c)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:36.630660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:36.681785 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2bbd936c1ec64798ad6557a5376cc32c', 'request_id': 'req-b714071a-d500-4d0c-af15-e8653c4401c5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.687397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2bbd936c1ec64798ad6557a5376cc32c', 'request_id': 'req-b714071a-d500-4d0c-af15-e8653c4401c5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.691030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:36.696428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2bbd936c1ec64798ad6557a5376cc32c', 'request_id': 'req-b714071a-d500-4d0c-af15-e8653c4401c5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.721039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] The token cache is being invalidate because project 2bbd936c1ec64798ad6557a5376cc32c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:36.721330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.722006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.723571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 997/1992] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:36 2022] DELETE /identity/v3/projects/2bbd936c1ec64798ad6557a5376cc32c => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:36.728631 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:36.730178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:36.745145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd166a0-0f54-4fb1-9a12-c35cbe33e3e8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:36.805982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:36.839467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:36.840570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:36.840984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:36.841293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] PATH_INFO: `/v3/users/ec74792aab614cf98073e8202a64088f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:36.843222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ec74792aab614cf98073e8202a64088f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:36.845578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:36.904843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ec74792aab614cf98073e8202a64088f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.918641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ec74792aab614cf98073e8202a64088f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.926549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ec74792aab614cf98073e8202a64088f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.929974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ec74792aab614cf98073e8202a64088f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.930680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7cc5a788-67f3-4146-9e7d-14f149074d7b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ec74792aab614cf98073e8202a64088f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:36.954872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 996/1993] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:36 2022] DELETE /identity/v3/users/ec74792aab614cf98073e8202a64088f => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:37.531619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:37.533738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:37.558874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b714071a-d500-4d0c-af15-e8653c4401c5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:37.680533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:37.681718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:37.682264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:37.682708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] PATH_INFO: `/v3/projects/2a1a14d3054e4e6e82c014ca047fd0b3` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:37.684428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a1a14d3054e4e6e82c014ca047fd0b3)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:37.686109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:37.746056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a1a14d3054e4e6e82c014ca047fd0b3', 'request_id': 'req-1f63cd70-c836-447d-b011-9e48e9ebf9e9'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:37.753265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a1a14d3054e4e6e82c014ca047fd0b3', 'request_id': 'req-1f63cd70-c836-447d-b011-9e48e9ebf9e9'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:37.764090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a1a14d3054e4e6e82c014ca047fd0b3', 'request_id': 'req-1f63cd70-c836-447d-b011-9e48e9ebf9e9'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:37.781237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] The token cache is being invalidate because project 2a1a14d3054e4e6e82c014ca047fd0b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:37.781937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:37.782997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f63cd70-c836-447d-b011-9e48e9ebf9e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:37.785319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 998/1994] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:37 2022] DELETE /identity/v3/projects/2a1a14d3054e4e6e82c014ca047fd0b3 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:37.792971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cac1a-c097-4264-a48a-9c0456f86162 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:37.793902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cac1a-c097-4264-a48a-9c0456f86162 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:37.794398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43cac1a-c097-4264-a48a-9c0456f86162 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:37.838779 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a43cac1a-c097-4264-a48a-9c0456f86162 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:37.840513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a43cac1a-c097-4264-a48a-9c0456f86162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:37.942566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 997/1995] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:37.944513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b38982-75be-4da0-b857-e8f8febe6f5a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:37.944920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b38982-75be-4da0-b857-e8f8febe6f5a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:37.945327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b38982-75be-4da0-b857-e8f8febe6f5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:37.981834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cac1a-c097-4264-a48a-9c0456f86162 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:37.981834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a43cac1a-c097-4264-a48a-9c0456f86162 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:37.992056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-42b38982-75be-4da0-b857-e8f8febe6f5a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:37.993336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-42b38982-75be-4da0-b857-e8f8febe6f5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:38.001560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43cac1a-c097-4264-a48a-9c0456f86162 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.007515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 999/1996] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:38.011283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.012509 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.014415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.015100 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] PATH_INFO: `/v3/users/7eb0a8c35ae641e382967dce8cf9fe65` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.019939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7eb0a8c35ae641e382967dce8cf9fe65)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.020085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b38982-75be-4da0-b857-e8f8febe6f5a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.023537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.031929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b38982-75be-4da0-b857-e8f8febe6f5a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.037702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae1be8-fc12-4818-a9d8-567ebb34eeaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.038557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae1be8-fc12-4818-a9d8-567ebb34eeaf admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.039152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae1be8-fc12-4818-a9d8-567ebb34eeaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.040090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdae1be8-fc12-4818-a9d8-567ebb34eeaf admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.041286 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae1be8-fc12-4818-a9d8-567ebb34eeaf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:38.041705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae1be8-fc12-4818-a9d8-567ebb34eeaf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.044527 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdae1be8-fc12-4818-a9d8-567ebb34eeaf admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.060295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1000/1997] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:38.065014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae1be8-fc12-4818-a9d8-567ebb34eeaf admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.077451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdae1be8-fc12-4818-a9d8-567ebb34eeaf admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.082446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29aef8-3f32-4921-a5d7-5c1d2d3e9884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.084341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29aef8-3f32-4921-a5d7-5c1d2d3e9884 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.084650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29aef8-3f32-4921-a5d7-5c1d2d3e9884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.084974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c29aef8-3f32-4921-a5d7-5c1d2d3e9884 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.085807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29aef8-3f32-4921-a5d7-5c1d2d3e9884 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.087978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c29aef8-3f32-4921-a5d7-5c1d2d3e9884 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.113385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7eb0a8c35ae641e382967dce8cf9fe65'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.119376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1001/1998] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:38 2022] POST /identity/v3/projects => generated 371 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:38.123516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29aef8-3f32-4921-a5d7-5c1d2d3e9884 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.127167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:38.155497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7eb0a8c35ae641e382967dce8cf9fe65'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.181872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7eb0a8c35ae641e382967dce8cf9fe65'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.182571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7eb0a8c35ae641e382967dce8cf9fe65'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.184805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-943e9c2a-08c7-4ef0-b7e8-d937f2d3ce51 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7eb0a8c35ae641e382967dce8cf9fe65'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.186409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:38.211988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 998/1999] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:37 2022] DELETE /identity/v3/users/7eb0a8c35ae641e382967dce8cf9fe65 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:38.216605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f140fc0-5669-4d70-a462-31eaf654576d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.217734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f140fc0-5669-4d70-a462-31eaf654576d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.218257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f140fc0-5669-4d70-a462-31eaf654576d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.220618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c29aef8-3f32-4921-a5d7-5c1d2d3e9884 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.237565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7f140fc0-5669-4d70-a462-31eaf654576d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:38.238831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7f140fc0-5669-4d70-a462-31eaf654576d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:38.299423 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c7b9-f158-4a14-af3b-ed2682946097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.300766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c7b9-f158-4a14-af3b-ed2682946097 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.301264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c7b9-f158-4a14-af3b-ed2682946097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.301768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b829c7b9-f158-4a14-af3b-ed2682946097 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.303505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829c7b9-f158-4a14-af3b-ed2682946097 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.307027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b829c7b9-f158-4a14-af3b-ed2682946097 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.311544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 999/2000] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:38.322016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f140fc0-5669-4d70-a462-31eaf654576d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.324163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7f140fc0-5669-4d70-a462-31eaf654576d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:38.343457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1002/2001] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:38 2022] POST /identity/v3/users => generated 286 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:38.347886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c7b9-f158-4a14-af3b-ed2682946097 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.356798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b829c7b9-f158-4a14-af3b-ed2682946097 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:38.372670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b829c7b9-f158-4a14-af3b-ed2682946097 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.377975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cb5c6-19fb-459a-a801-e60d401052c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.378667 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cb5c6-19fb-459a-a801-e60d401052c0 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.379011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cb5c6-19fb-459a-a801-e60d401052c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.379301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987cb5c6-19fb-459a-a801-e60d401052c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.380060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cb5c6-19fb-459a-a801-e60d401052c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.382671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987cb5c6-19fb-459a-a801-e60d401052c0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.394013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1003/2002] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:38.399602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cb5c6-19fb-459a-a801-e60d401052c0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.411205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f140fc0-5669-4d70-a462-31eaf654576d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.416179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987cb5c6-19fb-459a-a801-e60d401052c0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.418901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65a28358-a19d-40d6-b5c7-3fb312f8b937 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.419558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a28358-a19d-40d6-b5c7-3fb312f8b937 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.419843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a28358-a19d-40d6-b5c7-3fb312f8b937 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.420116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65a28358-a19d-40d6-b5c7-3fb312f8b937 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.421795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27164a85-785c-4e10-b12b-5bc3d6b24cb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.422495 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27164a85-785c-4e10-b12b-5bc3d6b24cb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.422819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27164a85-785c-4e10-b12b-5bc3d6b24cb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.423123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27164a85-785c-4e10-b12b-5bc3d6b24cb3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.423844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27164a85-785c-4e10-b12b-5bc3d6b24cb3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.425469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27164a85-785c-4e10-b12b-5bc3d6b24cb3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.435730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a28358-a19d-40d6-b5c7-3fb312f8b937 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.437085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65a28358-a19d-40d6-b5c7-3fb312f8b937 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.438290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1004/2003] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:38 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:38.442566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27164a85-785c-4e10-b12b-5bc3d6b24cb3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.447965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1000/2004] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:38.453296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27164a85-785c-4e10-b12b-5bc3d6b24cb3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.458558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6afcd-d66e-4065-bbde-95582a0af6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.459675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6afcd-d66e-4065-bbde-95582a0af6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.459982 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6afcd-d66e-4065-bbde-95582a0af6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.460324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c6afcd-d66e-4065-bbde-95582a0af6ec admin admin] PATH_INFO: `/v3/projects/e2814c3434194acb91f194c54237f27b/users/934cc971fd5e4436a4db7d0cfe0efffe/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.477926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6afcd-d66e-4065-bbde-95582a0af6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2814c3434194acb91f194c54237f27b, user_id=934cc971fd5e4436a4db7d0cfe0efffe, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.480708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c6afcd-d66e-4065-bbde-95582a0af6ec admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.496247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1005/2005] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:38 2022] PUT /identity/v3/projects/e2814c3434194acb91f194c54237f27b/users/934cc971fd5e4436a4db7d0cfe0efffe/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:38.501984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaad3a-8f1c-4500-8ea9-0f6409821e38 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.502601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaad3a-8f1c-4500-8ea9-0f6409821e38 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.502921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addaad3a-8f1c-4500-8ea9-0f6409821e38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.526436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-addaad3a-8f1c-4500-8ea9-0f6409821e38 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:38.527668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-addaad3a-8f1c-4500-8ea9-0f6409821e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:38.615654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1001/2006] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:38.630697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6afcd-d66e-4065-bbde-95582a0af6ec admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.633880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e7c6afcd-d66e-4065-bbde-95582a0af6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:38.721330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c6afcd-d66e-4065-bbde-95582a0af6ec admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.726002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d53c1-62bd-43d7-9c25-bbcc3a608be8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.726667 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d53c1-62bd-43d7-9c25-bbcc3a608be8 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.726956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d53c1-62bd-43d7-9c25-bbcc3a608be8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.727229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a98d53c1-62bd-43d7-9c25-bbcc3a608be8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.736703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d53c1-62bd-43d7-9c25-bbcc3a608be8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.738175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a98d53c1-62bd-43d7-9c25-bbcc3a608be8 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.750647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1006/2007] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:38.788509 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaad3a-8f1c-4500-8ea9-0f6409821e38 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.790111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-addaad3a-8f1c-4500-8ea9-0f6409821e38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:38.814887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addaad3a-8f1c-4500-8ea9-0f6409821e38 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:38.826018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:38.826941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:38.827229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:38.827621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] PATH_INFO: `/v3/projects/7e1e312d370247289f51922a891919b0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:38.829079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e1e312d370247289f51922a891919b0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:38.831300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:38.840623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:38.920771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e1e312d370247289f51922a891919b0', 'request_id': 'req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.928928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e1e312d370247289f51922a891919b0', 'request_id': 'req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.945550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e1e312d370247289f51922a891919b0', 'request_id': 'req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.964729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] The token cache is being invalidate because project 7e1e312d370247289f51922a891919b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:38.964945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.965667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:38.967523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1002/2008] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:38 2022] DELETE /identity/v3/projects/7e1e312d370247289f51922a891919b0 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:38.971737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d53c1-62bd-43d7-9c25-bbcc3a608be8 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:38.984328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a98d53c1-62bd-43d7-9c25-bbcc3a608be8 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:39.045741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:39.082272 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:39.083225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:39.083511 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:39.084394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] PATH_INFO: `/v3/users/43a96ed742e94e248ddf77d57cec1915` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:39.085697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43a96ed742e94e248ddf77d57cec1915)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:39.087685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:39.180008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43a96ed742e94e248ddf77d57cec1915'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.205469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43a96ed742e94e248ddf77d57cec1915'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.216526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43a96ed742e94e248ddf77d57cec1915'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.217627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43a96ed742e94e248ddf77d57cec1915'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.218310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4a7d9ad9-700c-4c35-af71-6509331254f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43a96ed742e94e248ddf77d57cec1915'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.236540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1007/2009] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:38 2022] DELETE /identity/v3/users/43a96ed742e94e248ddf77d57cec1915 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:39.514816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:39.516248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:39.531417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b04a34ec-704e-4e3b-ba99-b11d7fc5eec3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:39.601449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:39.603207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:39.603506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:39.603786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] PATH_INFO: `/v3/users/30a9be9e06e54718bc157486848fceca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:39.605773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=30a9be9e06e54718bc157486848fceca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:39.608775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:39.680371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '30a9be9e06e54718bc157486848fceca'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.697315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '30a9be9e06e54718bc157486848fceca'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.706710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '30a9be9e06e54718bc157486848fceca'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.707556 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '30a9be9e06e54718bc157486848fceca'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.708503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '30a9be9e06e54718bc157486848fceca'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:39.723148 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1003/2010] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:39 2022] DELETE /identity/v3/users/30a9be9e06e54718bc157486848fceca => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:39.727673 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc091f0f-e324-4949-bb2d-6b1e6e1f7327 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:39.728419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc091f0f-e324-4949-bb2d-6b1e6e1f7327 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:39.728419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc091f0f-e324-4949-bb2d-6b1e6e1f7327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:39.747545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cc091f0f-e324-4949-bb2d-6b1e6e1f7327 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:39.748773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cc091f0f-e324-4949-bb2d-6b1e6e1f7327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:39.830939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:39.832985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:39.873880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1008/2011] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:39.884232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc091f0f-e324-4949-bb2d-6b1e6e1f7327 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:39.886557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cc091f0f-e324-4949-bb2d-6b1e6e1f7327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:39.904550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:39.980714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc091f0f-e324-4949-bb2d-6b1e6e1f7327 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:39.984416 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e7a609-25b7-42f4-8edf-3254bc716f9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:39.985085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7a609-25b7-42f4-8edf-3254bc716f9f service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:39.985372 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7a609-25b7-42f4-8edf-3254bc716f9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:39.985687 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e7a609-25b7-42f4-8edf-3254bc716f9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:39.995030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:40.000002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7a609-25b7-42f4-8edf-3254bc716f9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.001386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e7a609-25b7-42f4-8edf-3254bc716f9f service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.020580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25262454-a824-42c4-8125-111fec4399d8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.028708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1009/2012] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:40.030236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b22f63-f0ee-45be-bb1c-de0a67e29a74 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.030597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b22f63-f0ee-45be-bb1c-de0a67e29a74 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.030948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10b22f63-f0ee-45be-bb1c-de0a67e29a74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.032156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.033946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.034643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.035336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] PATH_INFO: `/v3/projects/2963c83f31cf4753b33bee82edc8716a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.038587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=2963c83f31cf4753b33bee82edc8716a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.042737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.097718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-10b22f63-f0ee-45be-bb1c-de0a67e29a74 None None] MFA Rules not processed for user `934cc971fd5e4436a4db7d0cfe0efffe`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:40.099995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-10b22f63-f0ee-45be-bb1c-de0a67e29a74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.168067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2963c83f31cf4753b33bee82edc8716a', 'request_id': 'req-dcb9cca1-d721-433c-a4bc-c6e7321818ef'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.184895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2963c83f31cf4753b33bee82edc8716a', 'request_id': 'req-dcb9cca1-d721-433c-a4bc-c6e7321818ef'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.198346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2963c83f31cf4753b33bee82edc8716a', 'request_id': 'req-dcb9cca1-d721-433c-a4bc-c6e7321818ef'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.232767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] The token cache is being invalidate because project 2963c83f31cf4753b33bee82edc8716a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:40.234382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.235889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:40.237848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1010/2013] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:40.240766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1004/2014] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:39 2022] DELETE /identity/v3/projects/2963c83f31cf4753b33bee82edc8716a => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:40.255055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.256510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35614-a425-4232-96d4-10b7a567df2f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.256945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.257132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35614-a425-4232-96d4-10b7a567df2f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.257377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01c35614-a425-4232-96d4-10b7a567df2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.284778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-01c35614-a425-4232-96d4-10b7a567df2f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:40.287732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-01c35614-a425-4232-96d4-10b7a567df2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.416651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb9cca1-d721-433c-a4bc-c6e7321818ef admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.422437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de38bd-b0e1-48f2-af78-c91233fc8f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.423202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de38bd-b0e1-48f2-af78-c91233fc8f1d admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.423990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de38bd-b0e1-48f2-af78-c91233fc8f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.423990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0de38bd-b0e1-48f2-af78-c91233fc8f1d admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.424827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de38bd-b0e1-48f2-af78-c91233fc8f1d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.427498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0de38bd-b0e1-48f2-af78-c91233fc8f1d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.439492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1011/2015] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:40.462513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9991b4-7d43-43bb-9437-d24120c846ac None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.462821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9991b4-7d43-43bb-9437-d24120c846ac None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.463106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9991b4-7d43-43bb-9437-d24120c846ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.496711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1005/2016] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/projects => generated 371 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:40.507117 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de38bd-b0e1-48f2-af78-c91233fc8f1d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.507967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8e9991b4-7d43-43bb-9437-d24120c846ac None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:40.511298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8e9991b4-7d43-43bb-9437-d24120c846ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.583099 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:40.629345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1012/2017] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:40.635097 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9991b4-7d43-43bb-9437-d24120c846ac None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.643478 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0de38bd-b0e1-48f2-af78-c91233fc8f1d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.650515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9991b4-7d43-43bb-9437-d24120c846ac None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.652807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d640f88-d3df-447d-8a5a-4c18940d334a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.653752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d640f88-d3df-447d-8a5a-4c18940d334a admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.654234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d640f88-d3df-447d-8a5a-4c18940d334a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.655467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d640f88-d3df-447d-8a5a-4c18940d334a admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.658134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d640f88-d3df-447d-8a5a-4c18940d334a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.659999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea23e6-d855-417c-9333-a85fe066b14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.662001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea23e6-d855-417c-9333-a85fe066b14f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.662496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea23e6-d855-417c-9333-a85fe066b14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.663012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aea23e6-d855-417c-9333-a85fe066b14f admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.663604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d640f88-d3df-447d-8a5a-4c18940d334a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.664551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea23e6-d855-417c-9333-a85fe066b14f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:40.664946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea23e6-d855-417c-9333-a85fe066b14f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.666883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aea23e6-d855-417c-9333-a85fe066b14f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.684247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1013/2018] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:24:40.690316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea23e6-d855-417c-9333-a85fe066b14f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.709010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1006/2019] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/users => generated 292 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:40.715094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d640f88-d3df-447d-8a5a-4c18940d334a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.718641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aea23e6-d855-417c-9333-a85fe066b14f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.733955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018cd0e-4726-49c8-8367-d5af471883e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.734651 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018cd0e-4726-49c8-8367-d5af471883e7 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.734917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018cd0e-4726-49c8-8367-d5af471883e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.735199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018cd0e-4726-49c8-8367-d5af471883e7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.736202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018cd0e-4726-49c8-8367-d5af471883e7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.738999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d640f88-d3df-447d-8a5a-4c18940d334a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.742697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018cd0e-4726-49c8-8367-d5af471883e7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.756477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785da2c-cf25-4cda-8cc4-4284fc2caf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.757720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785da2c-cf25-4cda-8cc4-4284fc2caf55 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.758034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785da2c-cf25-4cda-8cc4-4284fc2caf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.758427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0785da2c-cf25-4cda-8cc4-4284fc2caf55 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.759345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785da2c-cf25-4cda-8cc4-4284fc2caf55 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.761066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0785da2c-cf25-4cda-8cc4-4284fc2caf55 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.766349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:40.790849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1007/2020] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:40.795704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785da2c-cf25-4cda-8cc4-4284fc2caf55 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.799281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1014/2021] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:40.800179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018cd0e-4726-49c8-8367-d5af471883e7 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.808437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0785da2c-cf25-4cda-8cc4-4284fc2caf55 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.887589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb76ca-5ce1-42e8-a9fb-f1fd18f41146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.888600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb76ca-5ce1-42e8-a9fb-f1fd18f41146 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.888925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb76ca-5ce1-42e8-a9fb-f1fd18f41146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.889298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41bb76ca-5ce1-42e8-a9fb-f1fd18f41146 admin admin] PATH_INFO: `/v3/projects/4487fb6f70e345d09d6b1ab426913c9a/users/8c1fe95525024eaf9dc3776c1dbcb6b7/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.905153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb76ca-5ce1-42e8-a9fb-f1fd18f41146 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4487fb6f70e345d09d6b1ab426913c9a, user_id=8c1fe95525024eaf9dc3776c1dbcb6b7, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.907874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41bb76ca-5ce1-42e8-a9fb-f1fd18f41146 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.913702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018cd0e-4726-49c8-8367-d5af471883e7 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:40.919971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1481d-2f06-408d-bb09-93d75d3c0dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:40.920730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1481d-2f06-408d-bb09-93d75d3c0dad admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:40.921095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1481d-2f06-408d-bb09-93d75d3c0dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:40.921441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce1481d-2f06-408d-bb09-93d75d3c0dad admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:40.922166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1481d-2f06-408d-bb09-93d75d3c0dad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:40.924084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1008/2022] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:40 2022] PUT /identity/v3/projects/4487fb6f70e345d09d6b1ab426913c9a/users/8c1fe95525024eaf9dc3776c1dbcb6b7/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:40.924200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce1481d-2f06-408d-bb09-93d75d3c0dad admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:40.925082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb76ca-5ce1-42e8-a9fb-f1fd18f41146 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.929462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-41bb76ca-5ce1-42e8-a9fb-f1fd18f41146 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:40.956041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1015/2023] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:40 2022] POST /identity/v3/users => generated 288 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:40.957800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1481d-2f06-408d-bb09-93d75d3c0dad admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:40.969812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce1481d-2f06-408d-bb09-93d75d3c0dad admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.000790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41bb76ca-5ce1-42e8-a9fb-f1fd18f41146 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.006620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.007492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.007837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.008061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] PATH_INFO: `/v3/projects/47775f27e33448059d70a822b0931022` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.009322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] RBAC: Authorizing `identity:delete_project(project_id=47775f27e33448059d70a822b0931022)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.011332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.035874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2a36b-df54-419e-bc64-6573e177b989 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.036562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2a36b-df54-419e-bc64-6573e177b989 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.036844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2a36b-df54-419e-bc64-6573e177b989 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.037196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4b2a36b-df54-419e-bc64-6573e177b989 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.037891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2a36b-df54-419e-bc64-6573e177b989 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.039524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4b2a36b-df54-419e-bc64-6573e177b989 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.053195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1016/2024] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:40 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:41.054341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2a36b-df54-419e-bc64-6573e177b989 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.065109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:INSERT=4,keystone:DELETE=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:41.065846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4b2a36b-df54-419e-bc64-6573e177b989 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.080813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47775f27e33448059d70a822b0931022', 'request_id': 'req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.087692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47775f27e33448059d70a822b0931022', 'request_id': 'req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.093808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47775f27e33448059d70a822b0931022', 'request_id': 'req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.107452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] The token cache is being invalidate because project 47775f27e33448059d70a822b0931022 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:41.107742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.108392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.110014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1009/2025] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:40 2022] DELETE /identity/v3/projects/47775f27e33448059d70a822b0931022 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:41.110756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.112356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.134947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cce5d-328e-43b2-8267-3d0fe797d2f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.135635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cce5d-328e-43b2-8267-3d0fe797d2f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.136418 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cce5d-328e-43b2-8267-3d0fe797d2f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.136802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-760cce5d-328e-43b2-8267-3d0fe797d2f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.137605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cce5d-328e-43b2-8267-3d0fe797d2f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.139078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-760cce5d-328e-43b2-8267-3d0fe797d2f1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.152168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1017/2026] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:41.153024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cce5d-328e-43b2-8267-3d0fe797d2f1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.154950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-760cce5d-328e-43b2-8267-3d0fe797d2f1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.168873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-760cce5d-328e-43b2-8267-3d0fe797d2f1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.176651 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.179080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.179448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.179970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] PATH_INFO: `/v3/users/5a1da32382cd484795961dc4f0d47e49` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.181638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a1da32382cd484795961dc4f0d47e49)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.184150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.196476 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:41.199763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a8e439-0c12-45ab-9fc2-4cba87d88aeb admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.204658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b2501-8e3f-4e08-98c4-08f2e0416af5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.205572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b2501-8e3f-4e08-98c4-08f2e0416af5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.205911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b2501-8e3f-4e08-98c4-08f2e0416af5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.206200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135b2501-8e3f-4e08-98c4-08f2e0416af5 admin admin] PATH_INFO: `/v3/projects/4487fb6f70e345d09d6b1ab426913c9a/users/8c1fe95525024eaf9dc3776c1dbcb6b7/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.208511 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b2501-8e3f-4e08-98c4-08f2e0416af5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4487fb6f70e345d09d6b1ab426913c9a, user_id=8c1fe95525024eaf9dc3776c1dbcb6b7, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.211320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135b2501-8e3f-4e08-98c4-08f2e0416af5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.226357 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1010/2027] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:41 2022] PUT /identity/v3/projects/4487fb6f70e345d09d6b1ab426913c9a/users/8c1fe95525024eaf9dc3776c1dbcb6b7/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:41.227231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b2501-8e3f-4e08-98c4-08f2e0416af5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.235423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-135b2501-8e3f-4e08-98c4-08f2e0416af5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.257666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a1da32382cd484795961dc4f0d47e49'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.274352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a1da32382cd484795961dc4f0d47e49'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.286149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a1da32382cd484795961dc4f0d47e49'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.288175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a1da32382cd484795961dc4f0d47e49'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.289112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a1da32382cd484795961dc4f0d47e49'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:41.306579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1018/2028] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:41 2022] DELETE /identity/v3/users/5a1da32382cd484795961dc4f0d47e49 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:41.334313 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135b2501-8e3f-4e08-98c4-08f2e0416af5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.419497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3311efb4-12e7-4e20-9936-fa3be9d0d200 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.420485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311efb4-12e7-4e20-9936-fa3be9d0d200 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.420974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311efb4-12e7-4e20-9936-fa3be9d0d200 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.421459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3311efb4-12e7-4e20-9936-fa3be9d0d200 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.422752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311efb4-12e7-4e20-9936-fa3be9d0d200 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.425047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3311efb4-12e7-4e20-9936-fa3be9d0d200 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.431270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:41.459223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1011/2029] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:41 2022] GET /identity/v3/roles => generated 1420 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:41.460765 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.463542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.488558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d944fa82-0f60-4cd2-9529-656f372d20f3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.498191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a4032-490e-4c0c-b89a-a9366fa23525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.499860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a4032-490e-4c0c-b89a-a9366fa23525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.500774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a4032-490e-4c0c-b89a-a9366fa23525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.501354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9a4032-490e-4c0c-b89a-a9366fa23525 admin admin] PATH_INFO: `/v3/projects/d7663cc6d2d141549698c621b24ec90e/users/192ba33fac934931bdc3046ced5e4e59/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.536013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a4032-490e-4c0c-b89a-a9366fa23525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d7663cc6d2d141549698c621b24ec90e, user_id=192ba33fac934931bdc3046ced5e4e59, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.541116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9a4032-490e-4c0c-b89a-a9366fa23525 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.574811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1019/2030] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:41 2022] PUT /identity/v3/projects/d7663cc6d2d141549698c621b24ec90e/users/192ba33fac934931bdc3046ced5e4e59/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:41.577227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5dadb-84a1-44fe-baed-e76653722fcd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.577834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5dadb-84a1-44fe-baed-e76653722fcd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.578576 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e5dadb-84a1-44fe-baed-e76653722fcd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.639189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-98e5dadb-84a1-44fe-baed-e76653722fcd None None] MFA Rules not processed for user `192ba33fac934931bdc3046ced5e4e59`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:41.641665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-98e5dadb-84a1-44fe-baed-e76653722fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.780566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1012/2031] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:41.792459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a4032-490e-4c0c-b89a-a9366fa23525 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:41.794368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-dd9a4032-490e-4c0c-b89a-a9366fa23525 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:41.945065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9a4032-490e-4c0c-b89a-a9366fa23525 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:41.950760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f2dd76-1291-4540-a2cb-f001b12889e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:41.951442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2dd76-1291-4540-a2cb-f001b12889e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:41.951773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2dd76-1291-4540-a2cb-f001b12889e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:41.952056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f2dd76-1291-4540-a2cb-f001b12889e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:41.964571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2dd76-1291-4540-a2cb-f001b12889e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:41.965604 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f2dd76-1291-4540-a2cb-f001b12889e9 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:41.979499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1020/2032] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:42.225091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5dadb-84a1-44fe-baed-e76653722fcd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.227904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-98e5dadb-84a1-44fe-baed-e76653722fcd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.415247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e5dadb-84a1-44fe-baed-e76653722fcd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.426603 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.428036 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.428656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.429091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] PATH_INFO: `/v3/projects/e839a5f0a3884f5ab6e3ba92587b7117` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.431606 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] RBAC: Authorizing `identity:delete_project(project_id=e839a5f0a3884f5ab6e3ba92587b7117)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.434056 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.525295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e839a5f0a3884f5ab6e3ba92587b7117', 'request_id': 'req-fa911681-ea5f-48cc-8002-0cfc78b2850d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.537426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e839a5f0a3884f5ab6e3ba92587b7117', 'request_id': 'req-fa911681-ea5f-48cc-8002-0cfc78b2850d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.545111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e839a5f0a3884f5ab6e3ba92587b7117', 'request_id': 'req-fa911681-ea5f-48cc-8002-0cfc78b2850d'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.586758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] The token cache is being invalidate because project e839a5f0a3884f5ab6e3ba92587b7117 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:42.587066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.597489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fa911681-ea5f-48cc-8002-0cfc78b2850d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:42.609557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1013/2033] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:42 2022] DELETE /identity/v3/projects/e839a5f0a3884f5ab6e3ba92587b7117 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:42.612489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c9325-5c89-47f4-9c63-f0080e23baca None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.613114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c9325-5c89-47f4-9c63-f0080e23baca None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.613543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-325c9325-5c89-47f4-9c63-f0080e23baca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.641280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-325c9325-5c89-47f4-9c63-f0080e23baca None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:42.643598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-325c9325-5c89-47f4-9c63-f0080e23baca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.749334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1021/2034] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:42.751044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9efc81-ddf7-4b83-9c17-f1965508b1a0 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.751496 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9efc81-ddf7-4b83-9c17-f1965508b1a0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.751901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb9efc81-ddf7-4b83-9c17-f1965508b1a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.776613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57358883-8ee6-4262-bdf1-2a956f4d8e70 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.777230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57358883-8ee6-4262-bdf1-2a956f4d8e70 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.777743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57358883-8ee6-4262-bdf1-2a956f4d8e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.778659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bb9efc81-ddf7-4b83-9c17-f1965508b1a0 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:42.780293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bb9efc81-ddf7-4b83-9c17-f1965508b1a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.787239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:42.796346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1014/2035] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:42.803214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9efc81-ddf7-4b83-9c17-f1965508b1a0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.826979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb9efc81-ddf7-4b83-9c17-f1965508b1a0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.832304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-57358883-8ee6-4262-bdf1-2a956f4d8e70 None None] MFA Rules not processed for user `8c1fe95525024eaf9dc3776c1dbcb6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:42.833806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-57358883-8ee6-4262-bdf1-2a956f4d8e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:42.835333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a515d39-b0cf-4ceb-aa51-895cabc41bdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.836402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515d39-b0cf-4ceb-aa51-895cabc41bdd admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.836854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515d39-b0cf-4ceb-aa51-895cabc41bdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.837309 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a515d39-b0cf-4ceb-aa51-895cabc41bdd admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.839340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a515d39-b0cf-4ceb-aa51-895cabc41bdd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:42.840050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a515d39-b0cf-4ceb-aa51-895cabc41bdd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.842559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a515d39-b0cf-4ceb-aa51-895cabc41bdd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.852139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:42.864803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1015/2036] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:24:42.866493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a515d39-b0cf-4ceb-aa51-895cabc41bdd admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.887479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a515d39-b0cf-4ceb-aa51-895cabc41bdd admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:42.894228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881d801-1ccf-4fa2-a162-1b3a7769d194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:42.895026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881d801-1ccf-4fa2-a162-1b3a7769d194 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:42.895442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881d801-1ccf-4fa2-a162-1b3a7769d194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:42.895861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2881d801-1ccf-4fa2-a162-1b3a7769d194 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:42.896796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881d801-1ccf-4fa2-a162-1b3a7769d194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:42.900055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2881d801-1ccf-4fa2-a162-1b3a7769d194 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:42.942231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1016/2037] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:42 2022] POST /identity/v3/projects => generated 363 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:42.946846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881d801-1ccf-4fa2-a162-1b3a7769d194 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.963696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1022/2038] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:42 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:42.978776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57358883-8ee6-4262-bdf1-2a956f4d8e70 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:42.981234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-57358883-8ee6-4262-bdf1-2a956f4d8e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:43.052858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2881d801-1ccf-4fa2-a162-1b3a7769d194 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.063037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57439e-0b24-4769-8115-eb98ee8003d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.064059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57439e-0b24-4769-8115-eb98ee8003d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.064386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57439e-0b24-4769-8115-eb98ee8003d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.064676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c57439e-0b24-4769-8115-eb98ee8003d3 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.065651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57439e-0b24-4769-8115-eb98ee8003d3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.068372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c57439e-0b24-4769-8115-eb98ee8003d3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.089875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57358883-8ee6-4262-bdf1-2a956f4d8e70 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.104084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1017/2039] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:42 2022] POST /identity/v3/users => generated 282 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:43.108003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57439e-0b24-4769-8115-eb98ee8003d3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.127205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c57439e-0b24-4769-8115-eb98ee8003d3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.137758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230126b-e06d-4265-846c-01bbb175b54b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.138979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230126b-e06d-4265-846c-01bbb175b54b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.139467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230126b-e06d-4265-846c-01bbb175b54b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.139976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4230126b-e06d-4265-846c-01bbb175b54b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.141021 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230126b-e06d-4265-846c-01bbb175b54b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.143049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4230126b-e06d-4265-846c-01bbb175b54b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.160726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1018/2040] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:43.162256 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230126b-e06d-4265-846c-01bbb175b54b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.163407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c0ea-c8d6-46d1-9fa9-4543609cb792 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.164012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c0ea-c8d6-46d1-9fa9-4543609cb792 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.164298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c0ea-c8d6-46d1-9fa9-4543609cb792 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.164577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e63c0ea-c8d6-46d1-9fa9-4543609cb792 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.166798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0e63c0ea-c8d6-46d1-9fa9-4543609cb792 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:43.175791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4230126b-e06d-4265-846c-01bbb175b54b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.183491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c3d3c-c27c-4739-88fc-70bb881c5098 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.184277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c3d3c-c27c-4739-88fc-70bb881c5098 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.184565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c3d3c-c27c-4739-88fc-70bb881c5098 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.184841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69c3d3c-c27c-4739-88fc-70bb881c5098 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.185616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c3d3c-c27c-4739-88fc-70bb881c5098 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.187917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69c3d3c-c27c-4739-88fc-70bb881c5098 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.207885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1019/2041] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:43 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:43.214808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c3d3c-c27c-4739-88fc-70bb881c5098 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.229787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69c3d3c-c27c-4739-88fc-70bb881c5098 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.242411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af80122-9a5d-40bf-a8c9-3442d3dc22a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.243353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af80122-9a5d-40bf-a8c9-3442d3dc22a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.243649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af80122-9a5d-40bf-a8c9-3442d3dc22a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.243931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af80122-9a5d-40bf-a8c9-3442d3dc22a8 admin admin] PATH_INFO: `/v3/projects/ac3b9ce513b64234b2b68781f8d21c6a/users/5faf583d242f4840a748c9d691b48193/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.266124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af80122-9a5d-40bf-a8c9-3442d3dc22a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac3b9ce513b64234b2b68781f8d21c6a, user_id=5faf583d242f4840a748c9d691b48193, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.269775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af80122-9a5d-40bf-a8c9-3442d3dc22a8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.293583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1020/2042] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:43 2022] PUT /identity/v3/projects/ac3b9ce513b64234b2b68781f8d21c6a/users/5faf583d242f4840a748c9d691b48193/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:43.307147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872579d-7e12-4dd2-ab21-35977166bf41 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.307887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872579d-7e12-4dd2-ab21-35977166bf41 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.308210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f872579d-7e12-4dd2-ab21-35977166bf41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.332463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63c0ea-c8d6-46d1-9fa9-4543609cb792 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.335562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e63c0ea-c8d6-46d1-9fa9-4543609cb792 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.343570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:43.346217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f872579d-7e12-4dd2-ab21-35977166bf41 None None] MFA Rules not processed for user `5faf583d242f4840a748c9d691b48193`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:43.347995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f872579d-7e12-4dd2-ab21-35977166bf41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:43.352993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1023/2043] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:42 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:43.457795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1021/2044] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:43 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:43.459152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c0ea-c8d6-46d1-9fa9-4543609cb792 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.478819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e63c0ea-c8d6-46d1-9fa9-4543609cb792 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.607838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c591c-7ebb-4668-abd0-0714970c66bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.609659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c591c-7ebb-4668-abd0-0714970c66bf admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.609959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c591c-7ebb-4668-abd0-0714970c66bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.613484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7c591c-7ebb-4668-abd0-0714970c66bf admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.618480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c591c-7ebb-4668-abd0-0714970c66bf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.621475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7c591c-7ebb-4668-abd0-0714970c66bf admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:43.754099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1024/2045] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:43 2022] POST /identity/v3/projects => generated 365 bytes in 293 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:43.756124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872579d-7e12-4dd2-ab21-35977166bf41 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:43.782085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f872579d-7e12-4dd2-ab21-35977166bf41 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:43.796010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:43.934513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b1eff-e913-4ec7-96d3-01feaea41972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:43.935548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b1eff-e913-4ec7-96d3-01feaea41972 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:43.935999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b1eff-e913-4ec7-96d3-01feaea41972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:43.936427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b1eff-e913-4ec7-96d3-01feaea41972 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:43.937302 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b1eff-e913-4ec7-96d3-01feaea41972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:43.939825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b1eff-e913-4ec7-96d3-01feaea41972 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.003164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c591c-7ebb-4668-abd0-0714970c66bf admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.003739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1022/2046] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:43 2022] POST /identity/v3/users => generated 289 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:44.017584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7c591c-7ebb-4668-abd0-0714970c66bf admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.025716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113670ae-bdf7-4332-a512-6415010feac6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.027890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113670ae-bdf7-4332-a512-6415010feac6 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.030109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113670ae-bdf7-4332-a512-6415010feac6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.030659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113670ae-bdf7-4332-a512-6415010feac6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.031598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113670ae-bdf7-4332-a512-6415010feac6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.033881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113670ae-bdf7-4332-a512-6415010feac6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.054627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b1eff-e913-4ec7-96d3-01feaea41972 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.054762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1025/2047] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:44.066650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b1eff-e913-4ec7-96d3-01feaea41972 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.074823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3be0b2-b80d-48c0-90ff-476013f4c39b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.076010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3be0b2-b80d-48c0-90ff-476013f4c39b admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.076327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3be0b2-b80d-48c0-90ff-476013f4c39b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.076417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3be0b2-b80d-48c0-90ff-476013f4c39b admin admin] PATH_INFO: `/v3/projects/4739b23b641f458d9da8048e745f0c6c/users/d9e0e58a186f4992a00e074374ff035c/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.097408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3be0b2-b80d-48c0-90ff-476013f4c39b admin admin] RBAC: Authorizing `identity:create_grant(project_id=4739b23b641f458d9da8048e745f0c6c, user_id=d9e0e58a186f4992a00e074374ff035c, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.101187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb3be0b2-b80d-48c0-90ff-476013f4c39b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.120948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1023/2048] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:44 2022] PUT /identity/v3/projects/4739b23b641f458d9da8048e745f0c6c/users/d9e0e58a186f4992a00e074374ff035c/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:44.124841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113670ae-bdf7-4332-a512-6415010feac6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.139799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113670ae-bdf7-4332-a512-6415010feac6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.226549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c523b2b8-80a9-4fe6-a9ec-c94777d6856f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.228390 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523b2b8-80a9-4fe6-a9ec-c94777d6856f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.229066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523b2b8-80a9-4fe6-a9ec-c94777d6856f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.230515 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c523b2b8-80a9-4fe6-a9ec-c94777d6856f admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.232454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523b2b8-80a9-4fe6-a9ec-c94777d6856f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.235500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c523b2b8-80a9-4fe6-a9ec-c94777d6856f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.253398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1026/2049] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:44 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:44.257659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3be0b2-b80d-48c0-90ff-476013f4c39b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.285064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb3be0b2-b80d-48c0-90ff-476013f4c39b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.298961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c20675-97f3-46e2-a8b5-50bde6ebfbbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.300151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c20675-97f3-46e2-a8b5-50bde6ebfbbf admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.300590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c20675-97f3-46e2-a8b5-50bde6ebfbbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.301043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c20675-97f3-46e2-a8b5-50bde6ebfbbf admin admin] PATH_INFO: `/v3/projects/4739b23b641f458d9da8048e745f0c6c/users/d9e0e58a186f4992a00e074374ff035c/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.306155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c20675-97f3-46e2-a8b5-50bde6ebfbbf admin admin] RBAC: Authorizing `identity:create_grant(project_id=4739b23b641f458d9da8048e745f0c6c, user_id=d9e0e58a186f4992a00e074374ff035c, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.308869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81c20675-97f3-46e2-a8b5-50bde6ebfbbf admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.336990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1024/2050] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:44 2022] PUT /identity/v3/projects/4739b23b641f458d9da8048e745f0c6c/users/d9e0e58a186f4992a00e074374ff035c/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:44.349575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29612e-09d0-451a-ab39-9862665d8a92 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.349875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29612e-09d0-451a-ab39-9862665d8a92 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.350177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f29612e-09d0-451a-ab39-9862665d8a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.414851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2f29612e-09d0-451a-ab39-9862665d8a92 None None] MFA Rules not processed for user `d9e0e58a186f4992a00e074374ff035c`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:44.416265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2f29612e-09d0-451a-ab39-9862665d8a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:44.573804 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:44.597783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1027/2051] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:44 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:44.610596 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c20675-97f3-46e2-a8b5-50bde6ebfbbf admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.613694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-81c20675-97f3-46e2-a8b5-50bde6ebfbbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:44.691754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c20675-97f3-46e2-a8b5-50bde6ebfbbf admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.696465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1615b-f579-4a5f-8597-008e87cb37e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.697261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1615b-f579-4a5f-8597-008e87cb37e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.698223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1615b-f579-4a5f-8597-008e87cb37e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.698223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a1615b-f579-4a5f-8597-008e87cb37e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.718292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1615b-f579-4a5f-8597-008e87cb37e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.721096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a1615b-f579-4a5f-8597-008e87cb37e0 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:44.734053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1025/2052] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:44 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:44.857375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29612e-09d0-451a-ab39-9862665d8a92 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:44.879807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f29612e-09d0-451a-ab39-9862665d8a92 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:44.884623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12204a9-d23e-40f4-a600-f204f229a9f3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:44.885201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12204a9-d23e-40f4-a600-f204f229a9f3 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:44.885548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12204a9-d23e-40f4-a600-f204f229a9f3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:44.885896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12204a9-d23e-40f4-a600-f204f229a9f3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:44.996331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12204a9-d23e-40f4-a600-f204f229a9f3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:44.998232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12204a9-d23e-40f4-a600-f204f229a9f3 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:45.022523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1028/2053] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:44 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:45.069043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1615b-f579-4a5f-8597-008e87cb37e0 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:45.079611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a1615b-f579-4a5f-8597-008e87cb37e0 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:45.085763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ceb880-b14a-4fac-b80c-3e495e4dffb3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:45.086724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ceb880-b14a-4fac-b80c-3e495e4dffb3 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:45.087547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ceb880-b14a-4fac-b80c-3e495e4dffb3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:45.087969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ceb880-b14a-4fac-b80c-3e495e4dffb3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:45.185035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ceb880-b14a-4fac-b80c-3e495e4dffb3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:45.186692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86ceb880-b14a-4fac-b80c-3e495e4dffb3 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:45.191845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:45.213334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1026/2054] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:45 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:45.753556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12204a9-d23e-40f4-a600-f204f229a9f3 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:45.755895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b12204a9-d23e-40f4-a600-f204f229a9f3 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:45.862443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12204a9-d23e-40f4-a600-f204f229a9f3 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:45.871239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:45.872226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:45.872426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:45.872705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] PATH_INFO: `/v3/users/192ba33fac934931bdc3046ced5e4e59` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:45.874710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=192ba33fac934931bdc3046ced5e4e59)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:45.877318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:45.940719 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '192ba33fac934931bdc3046ced5e4e59'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:45.955276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '192ba33fac934931bdc3046ced5e4e59'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:45.969319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '192ba33fac934931bdc3046ced5e4e59'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:45.970155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '192ba33fac934931bdc3046ced5e4e59'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:45.970779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '192ba33fac934931bdc3046ced5e4e59'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:45.998156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1029/2055] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:45 2022] DELETE /identity/v3/users/192ba33fac934931bdc3046ced5e4e59 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:46.000126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bce523-4275-496f-8d91-0bcb27c1cd4e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:46.000614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bce523-4275-496f-8d91-0bcb27c1cd4e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:46.000712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15bce523-4275-496f-8d91-0bcb27c1cd4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:46.027212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-15bce523-4275-496f-8d91-0bcb27c1cd4e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:46.028565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-15bce523-4275-496f-8d91-0bcb27c1cd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:46.106572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1027/2056] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:46.117231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:46.118684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:46.136255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:46.207671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88fb1612-4866-44a8-8178-fd1a602b23e8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:46.213159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b3123d-e511-4b2b-a87d-10d17554ceaa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:46.213663 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3123d-e511-4b2b-a87d-10d17554ceaa service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:46.213967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3123d-e511-4b2b-a87d-10d17554ceaa service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:46.214257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b3123d-e511-4b2b-a87d-10d17554ceaa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:46.229040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3123d-e511-4b2b-a87d-10d17554ceaa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:46.231039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b3123d-e511-4b2b-a87d-10d17554ceaa service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:46.268992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1030/2057] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:47.323460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bce523-4275-496f-8d91-0bcb27c1cd4e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:47.324976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-15bce523-4275-496f-8d91-0bcb27c1cd4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:47.347237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15bce523-4275-496f-8d91-0bcb27c1cd4e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:47.362200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:47.363700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:47.363984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:47.365070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] PATH_INFO: `/v3/projects/d7663cc6d2d141549698c621b24ec90e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:47.370351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d7663cc6d2d141549698c621b24ec90e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:47.372294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:47.524276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd7663cc6d2d141549698c621b24ec90e', 'request_id': 'req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.532143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd7663cc6d2d141549698c621b24ec90e', 'request_id': 'req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.540571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd7663cc6d2d141549698c621b24ec90e', 'request_id': 'req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.560876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] The token cache is being invalidate because project d7663cc6d2d141549698c621b24ec90e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:47.561671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.562520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-bcefd1a5-9f4e-4bbe-a1f1-0f3fbb8dab33 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:47.568762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1028/2058] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:47 2022] DELETE /identity/v3/projects/d7663cc6d2d141549698c621b24ec90e => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:47.571280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5461-bc9f-4816-8918-c3d71b76b722 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:47.572226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5461-bc9f-4816-8918-c3d71b76b722 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:47.573129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6f5461-bc9f-4816-8918-c3d71b76b722 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:47.597970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7a6f5461-bc9f-4816-8918-c3d71b76b722 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:47.600428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7a6f5461-bc9f-4816-8918-c3d71b76b722 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:47.673796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1031/2059] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:47.683783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6de2e-4cfd-486a-a999-c47a33a71a70 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:47.684229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6de2e-4cfd-486a-a999-c47a33a71a70 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:47.684612 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a6de2e-4cfd-486a-a999-c47a33a71a70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:47.706564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b8a6de2e-4cfd-486a-a999-c47a33a71a70 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:47.709641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b8a6de2e-4cfd-486a-a999-c47a33a71a70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:47.721924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1029/2060] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:47.723828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5461-bc9f-4816-8918-c3d71b76b722 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:47.734694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6f5461-bc9f-4816-8918-c3d71b76b722 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:47.740597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363dfd3-f2eb-4461-ad5d-7e9719f4cfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:47.741273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363dfd3-f2eb-4461-ad5d-7e9719f4cfc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:47.741589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363dfd3-f2eb-4461-ad5d-7e9719f4cfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:47.741876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f363dfd3-f2eb-4461-ad5d-7e9719f4cfc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:47.742611 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363dfd3-f2eb-4461-ad5d-7e9719f4cfc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:47.742922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363dfd3-f2eb-4461-ad5d-7e9719f4cfc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:47.744575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f363dfd3-f2eb-4461-ad5d-7e9719f4cfc3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:47.757445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1032/2061] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:47.761573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6de2e-4cfd-486a-a999-c47a33a71a70 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:47.770913 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a6de2e-4cfd-486a-a999-c47a33a71a70 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:47.776019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5e7ed-eb1e-48c2-af69-c0144bf07ec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:47.776885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5e7ed-eb1e-48c2-af69-c0144bf07ec5 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:47.777009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5e7ed-eb1e-48c2-af69-c0144bf07ec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:47.777217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5e7ed-eb1e-48c2-af69-c0144bf07ec5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:47.777976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5e7ed-eb1e-48c2-af69-c0144bf07ec5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:47.780105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5e7ed-eb1e-48c2-af69-c0144bf07ec5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:47.811514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1030/2062] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:47 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:47.812790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363dfd3-f2eb-4461-ad5d-7e9719f4cfc3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:47.826823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f363dfd3-f2eb-4461-ad5d-7e9719f4cfc3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:47.894292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710fb7-d790-42fc-92fa-8322d9b29f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:47.894967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710fb7-d790-42fc-92fa-8322d9b29f34 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:47.895279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710fb7-d790-42fc-92fa-8322d9b29f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:47.895557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77710fb7-d790-42fc-92fa-8322d9b29f34 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:47.896257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77710fb7-d790-42fc-92fa-8322d9b29f34 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:47.898347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77710fb7-d790-42fc-92fa-8322d9b29f34 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:47.918953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1033/2063] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:47 2022] POST /identity/v3/users => generated 284 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:47.923092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5e7ed-eb1e-48c2-af69-c0144bf07ec5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:47.935169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5e7ed-eb1e-48c2-af69-c0144bf07ec5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:47.939911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8d747-56e8-4231-96ae-6d49d34a37ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:47.940585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8d747-56e8-4231-96ae-6d49d34a37ed admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:47.940887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8d747-56e8-4231-96ae-6d49d34a37ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:47.941143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f8d747-56e8-4231-96ae-6d49d34a37ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:47.941900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8d747-56e8-4231-96ae-6d49d34a37ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:47.944672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f8d747-56e8-4231-96ae-6d49d34a37ed admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:47.956759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1031/2064] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:47.961345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710fb7-d790-42fc-92fa-8322d9b29f34 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:47.971926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77710fb7-d790-42fc-92fa-8322d9b29f34 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:47.977444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a940d56-bd90-4242-9150-8427f47afb19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:47.978158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a940d56-bd90-4242-9150-8427f47afb19 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:47.978445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a940d56-bd90-4242-9150-8427f47afb19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:47.978726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a940d56-bd90-4242-9150-8427f47afb19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:47.979470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a940d56-bd90-4242-9150-8427f47afb19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:47.981095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a940d56-bd90-4242-9150-8427f47afb19 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:47.997558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1034/2065] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:48.002827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8d747-56e8-4231-96ae-6d49d34a37ed admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:48.018087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f8d747-56e8-4231-96ae-6d49d34a37ed admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:48.024047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257590a-9c22-460c-a811-94bb4c796630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:48.025155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257590a-9c22-460c-a811-94bb4c796630 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.025503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257590a-9c22-460c-a811-94bb4c796630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.026430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4257590a-9c22-460c-a811-94bb4c796630 admin admin] PATH_INFO: `/v3/projects/12297d403c7f4d2cbd9b2e5bbc570128/users/f8962313b5444eaebb2d5e5d982e0b29/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.068227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257590a-9c22-460c-a811-94bb4c796630 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12297d403c7f4d2cbd9b2e5bbc570128, user_id=f8962313b5444eaebb2d5e5d982e0b29, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:48.071442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4257590a-9c22-460c-a811-94bb4c796630 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:48.090933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1032/2066] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:48 2022] PUT /identity/v3/projects/12297d403c7f4d2cbd9b2e5bbc570128/users/f8962313b5444eaebb2d5e5d982e0b29/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:48.102792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ca031-33fc-4db1-b1b9-c6715a2184e1 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.103277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ca031-33fc-4db1-b1b9-c6715a2184e1 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.103704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e2ca031-33fc-4db1-b1b9-c6715a2184e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.145024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2e2ca031-33fc-4db1-b1b9-c6715a2184e1 None None] MFA Rules not processed for user `f8962313b5444eaebb2d5e5d982e0b29`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:48.147041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2e2ca031-33fc-4db1-b1b9-c6715a2184e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:48.158602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:48.205319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257590a-9c22-460c-a811-94bb4c796630 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:48.207456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4257590a-9c22-460c-a811-94bb4c796630 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:48.232945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:48.237065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1035/2067] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:48 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:48.249805 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ca031-33fc-4db1-b1b9-c6715a2184e1 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:48.251376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2e2ca031-33fc-4db1-b1b9-c6715a2184e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:48.347758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4257590a-9c22-460c-a811-94bb4c796630 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:48.363101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:48.364871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.365462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.365929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] PATH_INFO: `/v3/users/5faf583d242f4840a748c9d691b48193` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.368534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5faf583d242f4840a748c9d691b48193)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:48.371908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:48.382974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e2ca031-33fc-4db1-b1b9-c6715a2184e1 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:48.401110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb97af6-ae3d-4226-882f-648a426cae2d service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e46ec9c31094d03b6a413e09bc9f699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:48.402229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97af6-ae3d-4226-882f-648a426cae2d service glance] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.402805 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97af6-ae3d-4226-882f-648a426cae2d service glance] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.403285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb97af6-ae3d-4226-882f-648a426cae2d service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.436841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb97af6-ae3d-4226-882f-648a426cae2d service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:48.438998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb97af6-ae3d-4226-882f-648a426cae2d service glance] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:48.452682 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1036/2068] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:48 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:24:48.487590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5faf583d242f4840a748c9d691b48193'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:48.501398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5faf583d242f4840a748c9d691b48193'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:48.510871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5faf583d242f4840a748c9d691b48193'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:48.511705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5faf583d242f4840a748c9d691b48193'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:48.512377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5faf583d242f4840a748c9d691b48193'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:48.529992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1033/2069] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:48 2022] DELETE /identity/v3/users/5faf583d242f4840a748c9d691b48193 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:24:48.534210 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae9ef8-5099-4af2-86d5-f8bd8ef7b628 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.534497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae9ef8-5099-4af2-86d5-f8bd8ef7b628 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.534800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ae9ef8-5099-4af2-86d5-f8bd8ef7b628 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.556346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c3ae9ef8-5099-4af2-86d5-f8bd8ef7b628 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:48.558042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c3ae9ef8-5099-4af2-86d5-f8bd8ef7b628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:48.624926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1037/2070] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:48.635329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:48.636764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:48.711660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3e60eb-55bb-4ce3-9445-13a8f10e99d0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:48.716501 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97309f83-9e8f-4b38-b9f3-e333b582bfb2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:48.717425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309f83-9e8f-4b38-b9f3-e333b582bfb2 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:48.717746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309f83-9e8f-4b38-b9f3-e333b582bfb2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:48.718143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97309f83-9e8f-4b38-b9f3-e333b582bfb2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:48.730869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97309f83-9e8f-4b38-b9f3-e333b582bfb2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:48.732304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97309f83-9e8f-4b38-b9f3-e333b582bfb2 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:48.743990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1034/2071] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:24:49.050708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae9ef8-5099-4af2-86d5-f8bd8ef7b628 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.054851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c3ae9ef8-5099-4af2-86d5-f8bd8ef7b628 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.074281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ae9ef8-5099-4af2-86d5-f8bd8ef7b628 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.079188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.080238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.080589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.080870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] PATH_INFO: `/v3/projects/ac3b9ce513b64234b2b68781f8d21c6a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.082120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac3b9ce513b64234b2b68781f8d21c6a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.085115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.120697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:49.134579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac3b9ce513b64234b2b68781f8d21c6a', 'request_id': 'req-0c1cef78-6950-4115-8c5b-0df7ca504e1f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.140300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac3b9ce513b64234b2b68781f8d21c6a', 'request_id': 'req-0c1cef78-6950-4115-8c5b-0df7ca504e1f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.147736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac3b9ce513b64234b2b68781f8d21c6a', 'request_id': 'req-0c1cef78-6950-4115-8c5b-0df7ca504e1f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.159245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] The token cache is being invalidate because project ac3b9ce513b64234b2b68781f8d21c6a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:49.159535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.160164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.162795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1038/2072] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:49 2022] DELETE /identity/v3/projects/ac3b9ce513b64234b2b68781f8d21c6a => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:49.166707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97309f83-9e8f-4b38-b9f3-e333b582bfb2 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.177533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97309f83-9e8f-4b38-b9f3-e333b582bfb2 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.215869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:49.236518 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.237386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.237768 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.238045 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] PATH_INFO: `/v3/users/d9e0e58a186f4992a00e074374ff035c` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.239946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9e0e58a186f4992a00e074374ff035c)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.241922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.289164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9e0e58a186f4992a00e074374ff035c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.301371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9e0e58a186f4992a00e074374ff035c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.308639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9e0e58a186f4992a00e074374ff035c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.309549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9e0e58a186f4992a00e074374ff035c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.310163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ee76a27c-d361-48a4-ba29-1d2b2b6ee2c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9e0e58a186f4992a00e074374ff035c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.325003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1035/2073] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:24:49 2022] DELETE /identity/v3/users/d9e0e58a186f4992a00e074374ff035c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:49.765871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:49.767587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:49.784172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1cef78-6950-4115-8c5b-0df7ca504e1f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:49.854060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:49.854964 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.855249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.856155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] PATH_INFO: `/v3/projects/4739b23b641f458d9da8048e745f0c6c` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:49.857608 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] RBAC: Authorizing `identity:delete_project(project_id=4739b23b641f458d9da8048e745f0c6c)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:49.859582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:49.936494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4739b23b641f458d9da8048e745f0c6c', 'request_id': 'req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.944273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4739b23b641f458d9da8048e745f0c6c', 'request_id': 'req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.953875 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4739b23b641f458d9da8048e745f0c6c', 'request_id': 'req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.976031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] The token cache is being invalidate because project 4739b23b641f458d9da8048e745f0c6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:24:49.977066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.978149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-7cc66965-a0de-44dd-b0a5-dfdd80a5e2de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:24:49.985607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1039/2074] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:24:49 2022] DELETE /identity/v3/projects/4739b23b641f458d9da8048e745f0c6c => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:49.985863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc73e-05e5-44cb-8119-b989f6d10b09 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:49.986372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc73e-05e5-44cb-8119-b989f6d10b09 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:49.986491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c3cc73e-05e5-44cb-8119-b989f6d10b09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.007858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6c3cc73e-05e5-44cb-8119-b989f6d10b09 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:50.010209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6c3cc73e-05e5-44cb-8119-b989f6d10b09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:50.096859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1036/2075] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:50.105715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a328e7-636d-43b7-8d48-393d8bfd4c23 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.105715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a328e7-636d-43b7-8d48-393d8bfd4c23 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.105715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05a328e7-636d-43b7-8d48-393d8bfd4c23 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.125188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-05a328e7-636d-43b7-8d48-393d8bfd4c23 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:50.126468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-05a328e7-636d-43b7-8d48-393d8bfd4c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:50.135890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1040/2076] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:50.140069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cc73e-05e5-44cb-8119-b989f6d10b09 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.150135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c3cc73e-05e5-44cb-8119-b989f6d10b09 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.155103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcc799-9c8f-4d7e-88ca-6f1485492cfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.155858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcc799-9c8f-4d7e-88ca-6f1485492cfb admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.156246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcc799-9c8f-4d7e-88ca-6f1485492cfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.156600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19fcc799-9c8f-4d7e-88ca-6f1485492cfb admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.157411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcc799-9c8f-4d7e-88ca-6f1485492cfb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:24:50.157850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcc799-9c8f-4d7e-88ca-6f1485492cfb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.159528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19fcc799-9c8f-4d7e-88ca-6f1485492cfb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.175995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1037/2077] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:24:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:24:50.177732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a328e7-636d-43b7-8d48-393d8bfd4c23 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.188191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05a328e7-636d-43b7-8d48-393d8bfd4c23 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.193734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911daa0-7635-404f-9cd5-e2e42ce2756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.194431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911daa0-7635-404f-9cd5-e2e42ce2756d admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.194713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911daa0-7635-404f-9cd5-e2e42ce2756d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.194989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1911daa0-7635-404f-9cd5-e2e42ce2756d admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.195696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911daa0-7635-404f-9cd5-e2e42ce2756d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.197723 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1911daa0-7635-404f-9cd5-e2e42ce2756d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.224011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1041/2078] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:50 2022] POST /identity/v3/projects => generated 375 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:24:50.228292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcc799-9c8f-4d7e-88ca-6f1485492cfb admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.239372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19fcc799-9c8f-4d7e-88ca-6f1485492cfb admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.286658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:50.301018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4ae26-5341-490c-9e4d-9a0930c99712 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.302349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4ae26-5341-490c-9e4d-9a0930c99712 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.302600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4ae26-5341-490c-9e4d-9a0930c99712 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.302872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d4ae26-5341-490c-9e4d-9a0930c99712 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.303566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4ae26-5341-490c-9e4d-9a0930c99712 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.305476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9d4ae26-5341-490c-9e4d-9a0930c99712 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.332128 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1038/2079] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:50 2022] POST /identity/v3/users => generated 288 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:50.333356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911daa0-7635-404f-9cd5-e2e42ce2756d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.343934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1911daa0-7635-404f-9cd5-e2e42ce2756d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.348610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b5793-65be-4e61-9249-b316a6d6577f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.349468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b5793-65be-4e61-9249-b316a6d6577f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.349739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b5793-65be-4e61-9249-b316a6d6577f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.350057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95b5793-65be-4e61-9249-b316a6d6577f admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.350788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b5793-65be-4e61-9249-b316a6d6577f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.352391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d95b5793-65be-4e61-9249-b316a6d6577f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.362737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1042/2080] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:50.367364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4ae26-5341-490c-9e4d-9a0930c99712 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.377316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d4ae26-5341-490c-9e4d-9a0930c99712 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.382688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450b7cb3-d323-4760-a228-2cff3afed682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.383340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b7cb3-d323-4760-a228-2cff3afed682 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.383627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b7cb3-d323-4760-a228-2cff3afed682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.383909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450b7cb3-d323-4760-a228-2cff3afed682 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.384638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b7cb3-d323-4760-a228-2cff3afed682 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.386293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450b7cb3-d323-4760-a228-2cff3afed682 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.399975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1039/2081] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:50.403772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b5793-65be-4e61-9249-b316a6d6577f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.416736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95b5793-65be-4e61-9249-b316a6d6577f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.421413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b290627-fbc7-4b04-bab5-bd5af6c81cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.422335 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b290627-fbc7-4b04-bab5-bd5af6c81cc9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.422618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b290627-fbc7-4b04-bab5-bd5af6c81cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.422894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b290627-fbc7-4b04-bab5-bd5af6c81cc9 admin admin] PATH_INFO: `/v3/projects/1dfb0029d2004a3da6df55eb8fc7b50f/users/bcc382b27abe4d9dbb354ca54ae7aca6/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.426069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:50.440085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b290627-fbc7-4b04-bab5-bd5af6c81cc9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1dfb0029d2004a3da6df55eb8fc7b50f, user_id=bcc382b27abe4d9dbb354ca54ae7aca6, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.442833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b290627-fbc7-4b04-bab5-bd5af6c81cc9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.455824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1043/2082] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:50 2022] PUT /identity/v3/projects/1dfb0029d2004a3da6df55eb8fc7b50f/users/bcc382b27abe4d9dbb354ca54ae7aca6/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:50.467416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b73fc2-c92f-4b03-b3ed-5e995e0f26d6 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.467702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b73fc2-c92f-4b03-b3ed-5e995e0f26d6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.467981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b73fc2-c92f-4b03-b3ed-5e995e0f26d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.495173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-03b73fc2-c92f-4b03-b3ed-5e995e0f26d6 None None] MFA Rules not processed for user `bcc382b27abe4d9dbb354ca54ae7aca6`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:50.496319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-03b73fc2-c92f-4b03-b3ed-5e995e0f26d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:50.564865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1040/2083] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:50 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:24:50.569839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b290627-fbc7-4b04-bab5-bd5af6c81cc9 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.632723 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b290627-fbc7-4b04-bab5-bd5af6c81cc9 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.637405 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b71430-2308-4e6a-93f1-75359dbbf955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.638098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b71430-2308-4e6a-93f1-75359dbbf955 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.638383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b71430-2308-4e6a-93f1-75359dbbf955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.638660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b71430-2308-4e6a-93f1-75359dbbf955 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.639383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b71430-2308-4e6a-93f1-75359dbbf955 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.641296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b71430-2308-4e6a-93f1-75359dbbf955 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.671809 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1044/2084] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:24:50 2022] POST /identity/v3/projects => generated 377 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:50.673825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b73fc2-c92f-4b03-b3ed-5e995e0f26d6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.689549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b73fc2-c92f-4b03-b3ed-5e995e0f26d6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.748202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46effd00-7af1-4974-a637-a395e6332398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.748794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46effd00-7af1-4974-a637-a395e6332398 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.749080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46effd00-7af1-4974-a637-a395e6332398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.749364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46effd00-7af1-4974-a637-a395e6332398 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.750085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46effd00-7af1-4974-a637-a395e6332398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.752017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46effd00-7af1-4974-a637-a395e6332398 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.776061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1041/2085] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:24:50 2022] POST /identity/v3/users => generated 295 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:24:50.777203 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b71430-2308-4e6a-93f1-75359dbbf955 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.788180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b71430-2308-4e6a-93f1-75359dbbf955 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.792769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6e891-7dc5-4407-af75-6f0f763a66f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.793427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6e891-7dc5-4407-af75-6f0f763a66f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.793739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6e891-7dc5-4407-af75-6f0f763a66f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.794017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf6e891-7dc5-4407-af75-6f0f763a66f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.794755 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6e891-7dc5-4407-af75-6f0f763a66f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.796335 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf6e891-7dc5-4407-af75-6f0f763a66f0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.812307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1045/2086] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:24:50.813759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46effd00-7af1-4974-a637-a395e6332398 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.824591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46effd00-7af1-4974-a637-a395e6332398 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.829384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cef02-ee49-46c8-a074-3dbbe9e95490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.830336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cef02-ee49-46c8-a074-3dbbe9e95490 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.831923 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cef02-ee49-46c8-a074-3dbbe9e95490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.831923 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052cef02-ee49-46c8-a074-3dbbe9e95490 admin admin] PATH_INFO: `/v3/projects/f94c9f5dfda9483e9b2cb200a45e5958/users/2fc635dea4e54dc5a7ecb8cac10d5884/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.857803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cef02-ee49-46c8-a074-3dbbe9e95490 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94c9f5dfda9483e9b2cb200a45e5958, user_id=2fc635dea4e54dc5a7ecb8cac10d5884, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.860594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052cef02-ee49-46c8-a074-3dbbe9e95490 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.884808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1042/2087] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:50 2022] PUT /identity/v3/projects/f94c9f5dfda9483e9b2cb200a45e5958/users/2fc635dea4e54dc5a7ecb8cac10d5884/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:50.888496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6e891-7dc5-4407-af75-6f0f763a66f0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:50.902226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf6e891-7dc5-4407-af75-6f0f763a66f0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:50.977181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0acb4e-328f-4e09-9e0f-a0a9a4955437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:50.978139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0acb4e-328f-4e09-9e0f-a0a9a4955437 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:50.978492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0acb4e-328f-4e09-9e0f-a0a9a4955437 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:50.980281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0acb4e-328f-4e09-9e0f-a0a9a4955437 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:50.980790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0acb4e-328f-4e09-9e0f-a0a9a4955437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:50.982550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0acb4e-328f-4e09-9e0f-a0a9a4955437 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:50.997245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1046/2088] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:24:50 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:24:50.998735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cef02-ee49-46c8-a074-3dbbe9e95490 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.011739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052cef02-ee49-46c8-a074-3dbbe9e95490 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.017477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f53a9d-130d-415a-a61b-30dd5971d458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.018551 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f53a9d-130d-415a-a61b-30dd5971d458 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.018899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f53a9d-130d-415a-a61b-30dd5971d458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.019242 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f53a9d-130d-415a-a61b-30dd5971d458 admin admin] PATH_INFO: `/v3/projects/f94c9f5dfda9483e9b2cb200a45e5958/users/2fc635dea4e54dc5a7ecb8cac10d5884/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.021701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f53a9d-130d-415a-a61b-30dd5971d458 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f94c9f5dfda9483e9b2cb200a45e5958, user_id=2fc635dea4e54dc5a7ecb8cac10d5884, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.024418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f53a9d-130d-415a-a61b-30dd5971d458 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.035131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1043/2089] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:24:50 2022] PUT /identity/v3/projects/f94c9f5dfda9483e9b2cb200a45e5958/users/2fc635dea4e54dc5a7ecb8cac10d5884/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:24:51.046327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af906dba-4ca3-4ecf-93e3-fa540e911960 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.046616 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af906dba-4ca3-4ecf-93e3-fa540e911960 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.046894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af906dba-4ca3-4ecf-93e3-fa540e911960 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.075227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-af906dba-4ca3-4ecf-93e3-fa540e911960 None None] MFA Rules not processed for user `2fc635dea4e54dc5a7ecb8cac10d5884`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:24:51.076470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-af906dba-4ca3-4ecf-93e3-fa540e911960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.124222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:51.151187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1047/2090] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:24:51 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:24:51.162499 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f53a9d-130d-415a-a61b-30dd5971d458 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:24:51.164659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-53f53a9d-130d-415a-a61b-30dd5971d458 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:24:51.174853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:24:51.226542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f53a9d-130d-415a-a61b-30dd5971d458 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:24:51.230220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe9c929-9f7f-41b6-b7e0-f111ed212ff6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:24:51.230894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9c929-9f7f-41b6-b7e0-f111ed212ff6 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:24:51.231170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9c929-9f7f-41b6-b7e0-f111ed212ff6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:24:51.231533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe9c929-9f7f-41b6-b7e0-f111ed212ff6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:24:51.294423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9c929-9f7f-41b6-b7e0-f111ed212ff6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:24:51.295872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe9c929-9f7f-41b6-b7e0-f111ed212ff6 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:24:51.305950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1044/2091] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:24:51 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:01.148213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=9 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:01.300762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=45 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:02.168223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af906dba-4ca3-4ecf-93e3-fa540e911960 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:02.169779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-af906dba-4ca3-4ecf-93e3-fa540e911960 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:02.182436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af906dba-4ca3-4ecf-93e3-fa540e911960 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:02.186061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ded4e50-a910-4a8d-a971-b9cb9bbcc1dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:02.186710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded4e50-a910-4a8d-a971-b9cb9bbcc1dd service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:02.186994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded4e50-a910-4a8d-a971-b9cb9bbcc1dd service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:02.187271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ded4e50-a910-4a8d-a971-b9cb9bbcc1dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:02.197270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded4e50-a910-4a8d-a971-b9cb9bbcc1dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:02.198839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ded4e50-a910-4a8d-a971-b9cb9bbcc1dd service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:02.209067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1048/2092] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:02 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:12.204584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:13.005672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcdac4-a1c2-4277-9073-547a6a5fd946 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:13.006515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcdac4-a1c2-4277-9073-547a6a5fd946 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:13.006726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fcdac4-a1c2-4277-9073-547a6a5fd946 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:13.008192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1045/2093] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:25:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:25:13.045610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29779c-7205-4175-9ca7-c40e0c614bc3 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:13.045896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29779c-7205-4175-9ca7-c40e0c614bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:13.046179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c29779c-7205-4175-9ca7-c40e0c614bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:13.067999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5c29779c-7205-4175-9ca7-c40e0c614bc3 None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:13.069245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5c29779c-7205-4175-9ca7-c40e0c614bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:13.146526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1049/2094] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:25:13 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:13.183775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcdac4-a1c2-4277-9073-547a6a5fd946 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:13.186044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-02fcdac4-a1c2-4277-9073-547a6a5fd946 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:13.204506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fcdac4-a1c2-4277-9073-547a6a5fd946 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:13.210568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6069f-584f-4d2e-b28d-93c8bf7baa79 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:13.211223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6069f-584f-4d2e-b28d-93c8bf7baa79 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:13.211507 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6069f-584f-4d2e-b28d-93c8bf7baa79 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:13.211786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ef6069f-584f-4d2e-b28d-93c8bf7baa79 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:13.221068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6069f-584f-4d2e-b28d-93c8bf7baa79 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:13.222683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ef6069f-584f-4d2e-b28d-93c8bf7baa79 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:13.234621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1046/2095] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:25:13.749371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29779c-7205-4175-9ca7-c40e0c614bc3 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:13.751385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5c29779c-7205-4175-9ca7-c40e0c614bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:13.836718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c29779c-7205-4175-9ca7-c40e0c614bc3 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:13.841309 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:13.842532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:13.842874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:13.843235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] PATH_INFO: `/v3/users/934cc971fd5e4436a4db7d0cfe0efffe` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:13.844519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] RBAC: Authorizing `identity:delete_user(user_id=934cc971fd5e4436a4db7d0cfe0efffe)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:13.846715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:13.913164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '934cc971fd5e4436a4db7d0cfe0efffe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:13.930897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '934cc971fd5e4436a4db7d0cfe0efffe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:13.940706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '934cc971fd5e4436a4db7d0cfe0efffe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:13.943086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '934cc971fd5e4436a4db7d0cfe0efffe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:13.958841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '934cc971fd5e4436a4db7d0cfe0efffe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:13.979354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1050/2096] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:13 2022] DELETE /identity/v3/users/934cc971fd5e4436a4db7d0cfe0efffe => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:14.299172 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6069f-584f-4d2e-b28d-93c8bf7baa79 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:14.313904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ef6069f-584f-4d2e-b28d-93c8bf7baa79 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:14.393091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:14.393996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:14.394278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:14.394553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] PATH_INFO: `/v3/projects/e2814c3434194acb91f194c54237f27b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:14.396438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2814c3434194acb91f194c54237f27b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:14.400394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:14.468535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2814c3434194acb91f194c54237f27b', 'request_id': 'req-538def56-4bbf-4e68-ae91-f2d2315f6df0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:14.475916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2814c3434194acb91f194c54237f27b', 'request_id': 'req-538def56-4bbf-4e68-ae91-f2d2315f6df0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:14.482204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2814c3434194acb91f194c54237f27b', 'request_id': 'req-538def56-4bbf-4e68-ae91-f2d2315f6df0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:14.496541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] The token cache is being invalidate because project e2814c3434194acb91f194c54237f27b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:14.496834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:14.497494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:14.499117 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1047/2097] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:14 2022] DELETE /identity/v3/projects/e2814c3434194acb91f194c54237f27b => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:14.503126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:14.504548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:14.517802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df282fb0-490f-4aef-8cd3-2e4f68ec9830 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:14.532098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:14.593808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:14.594702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:14.595189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:14.595951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] PATH_INFO: `/v3/users/8c1fe95525024eaf9dc3776c1dbcb6b7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:14.596928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c1fe95525024eaf9dc3776c1dbcb6b7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:14.599664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:14.675707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c1fe95525024eaf9dc3776c1dbcb6b7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:14.690157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c1fe95525024eaf9dc3776c1dbcb6b7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:14.700677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c1fe95525024eaf9dc3776c1dbcb6b7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:14.701435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c1fe95525024eaf9dc3776c1dbcb6b7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:14.702112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-105944bf-a3de-4311-a792-ce2a771c029e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c1fe95525024eaf9dc3776c1dbcb6b7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:14.722968 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1051/2098] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:14 2022] DELETE /identity/v3/users/8c1fe95525024eaf9dc3776c1dbcb6b7 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:15.110550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:15.112135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:15.130049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-538def56-4bbf-4e68-ae91-f2d2315f6df0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:15.209688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:15.211025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.211415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.211835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] PATH_INFO: `/v3/projects/4487fb6f70e345d09d6b1ab426913c9a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.213153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4487fb6f70e345d09d6b1ab426913c9a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:15.215209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:15.238574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:15.265948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4487fb6f70e345d09d6b1ab426913c9a', 'request_id': 'req-80e3996f-fbd9-4881-b59c-a7a99c2f7128'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.272942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4487fb6f70e345d09d6b1ab426913c9a', 'request_id': 'req-80e3996f-fbd9-4881-b59c-a7a99c2f7128'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.281098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4487fb6f70e345d09d6b1ab426913c9a', 'request_id': 'req-80e3996f-fbd9-4881-b59c-a7a99c2f7128'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.298569 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] The token cache is being invalidate because project 4487fb6f70e345d09d6b1ab426913c9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:15.298922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.299595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-80e3996f-fbd9-4881-b59c-a7a99c2f7128 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.301543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1048/2099] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:15 2022] DELETE /identity/v3/projects/4487fb6f70e345d09d6b1ab426913c9a => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:15.307319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a161b-083c-45ae-af12-1f6bd89e1961 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.307686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8b371-5035-4d4c-901e-833f0f5921d4 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.308116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a161b-083c-45ae-af12-1f6bd89e1961 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.308236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8b371-5035-4d4c-901e-833f0f5921d4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.308336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f4a161b-083c-45ae-af12-1f6bd89e1961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.308596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c8b371-5035-4d4c-901e-833f0f5921d4 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.310528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1052/2100] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:25:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:25:15.318751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccc129-50ea-4f48-a997-928bd25fe10d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.319297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccc129-50ea-4f48-a997-928bd25fe10d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.319809 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bccc129-50ea-4f48-a997-928bd25fe10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.330357 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7f4a161b-083c-45ae-af12-1f6bd89e1961 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:15.332512 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7f4a161b-083c-45ae-af12-1f6bd89e1961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:15.344821 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8bccc129-50ea-4f48-a997-928bd25fe10d None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:15.346093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8bccc129-50ea-4f48-a997-928bd25fe10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:15.407817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1049/2101] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:15.408715 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a161b-083c-45ae-af12-1f6bd89e1961 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:15.410003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7f4a161b-083c-45ae-af12-1f6bd89e1961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:15.423044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f4a161b-083c-45ae-af12-1f6bd89e1961 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:15.428321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:15.428833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1053/2102] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:25:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:15.429349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.429464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.429777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] PATH_INFO: `/v3/users/beb901faa8e9410484f83e47e5bcfa8c` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.430456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0d756-41a1-4ef3-bee1-7dbdf69fafa4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.430810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0d756-41a1-4ef3-bee1-7dbdf69fafa4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.431002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=beb901faa8e9410484f83e47e5bcfa8c)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:15.431209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff0d756-41a1-4ef3-bee1-7dbdf69fafa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.432934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:15.456056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-aff0d756-41a1-4ef3-bee1-7dbdf69fafa4 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:15.457347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aff0d756-41a1-4ef3-bee1-7dbdf69fafa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:15.468800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1054/2103] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:15.470536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0d756-41a1-4ef3-bee1-7dbdf69fafa4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:15.473148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aff0d756-41a1-4ef3-bee1-7dbdf69fafa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:15.489750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff0d756-41a1-4ef3-bee1-7dbdf69fafa4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:15.495590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91318ea0-c0fa-4aa6-8bb8-a3d94cfb888e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:15.496308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91318ea0-c0fa-4aa6-8bb8-a3d94cfb888e service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.496603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91318ea0-c0fa-4aa6-8bb8-a3d94cfb888e service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.496853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'beb901faa8e9410484f83e47e5bcfa8c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.496970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91318ea0-c0fa-4aa6-8bb8-a3d94cfb888e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.507125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91318ea0-c0fa-4aa6-8bb8-a3d94cfb888e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:15.508752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91318ea0-c0fa-4aa6-8bb8-a3d94cfb888e service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:15.514860 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'beb901faa8e9410484f83e47e5bcfa8c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.519603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1055/2104] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:15.521310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91318ea0-c0fa-4aa6-8bb8-a3d94cfb888e service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:15.522484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'beb901faa8e9410484f83e47e5bcfa8c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.523243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'beb901faa8e9410484f83e47e5bcfa8c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.523948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'beb901faa8e9410484f83e47e5bcfa8c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.532840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91318ea0-c0fa-4aa6-8bb8-a3d94cfb888e service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:15.539286 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656516d-b9ee-4d1c-92f9-f489db58ab12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:15.540136 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656516d-b9ee-4d1c-92f9-f489db58ab12 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.540576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656516d-b9ee-4d1c-92f9-f489db58ab12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.540849 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656516d-b9ee-4d1c-92f9-f489db58ab12 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.542010 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656516d-b9ee-4d1c-92f9-f489db58ab12 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:15.542318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656516d-b9ee-4d1c-92f9-f489db58ab12 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:15.544199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656516d-b9ee-4d1c-92f9-f489db58ab12 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:15.546098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1050/2105] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:15 2022] DELETE /identity/v3/users/beb901faa8e9410484f83e47e5bcfa8c => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:15.551424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:15.560177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1056/2106] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:15.562729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:15.564335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:15.587312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:15.636701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7d414d-b69e-430e-86f7-1f0f0e00a5f4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:15.641329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c515f-4e2d-4486-8b79-74a751b89166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:15.642011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c515f-4e2d-4486-8b79-74a751b89166 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.642294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c515f-4e2d-4486-8b79-74a751b89166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.642568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579c515f-4e2d-4486-8b79-74a751b89166 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.643294 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c515f-4e2d-4486-8b79-74a751b89166 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:15.645214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579c515f-4e2d-4486-8b79-74a751b89166 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:15.671074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1051/2107] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:15 2022] POST /identity/v3/projects => generated 373 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:25:15.675251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656516d-b9ee-4d1c-92f9-f489db58ab12 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:15.686282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656516d-b9ee-4d1c-92f9-f489db58ab12 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:15.730743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c515f-4e2d-4486-8b79-74a751b89166 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:15.732852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-579c515f-4e2d-4486-8b79-74a751b89166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:15.756795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf39b7-9340-410c-ba56-9e5d71beb840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:15.757644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf39b7-9340-410c-ba56-9e5d71beb840 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.757953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf39b7-9340-410c-ba56-9e5d71beb840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.758136 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf39b7-9340-410c-ba56-9e5d71beb840 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.758812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf39b7-9340-410c-ba56-9e5d71beb840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:15.760756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcdf39b7-9340-410c-ba56-9e5d71beb840 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:15.783024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1057/2108] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:15 2022] POST /identity/v3/users => generated 287 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:15.787008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf39b7-9340-410c-ba56-9e5d71beb840 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:15.798298 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf39b7-9340-410c-ba56-9e5d71beb840 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:15.803553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d4a28-c66a-4cf1-a944-042474eaea74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:15.804206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d4a28-c66a-4cf1-a944-042474eaea74 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.804490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d4a28-c66a-4cf1-a944-042474eaea74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.804769 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d01d4a28-c66a-4cf1-a944-042474eaea74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.805490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d4a28-c66a-4cf1-a944-042474eaea74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:15.807112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d01d4a28-c66a-4cf1-a944-042474eaea74 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:15.815084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579c515f-4e2d-4486-8b79-74a751b89166 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:15.817108 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1058/2109] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:15.821463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:15.821743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d4a28-c66a-4cf1-a944-042474eaea74 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:15.822362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.822652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.822932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] PATH_INFO: `/v3/projects/05e0691b14804cc5b4fe120fc5dea871` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.824134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=05e0691b14804cc5b4fe120fc5dea871)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:15.826086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:15.831822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d01d4a28-c66a-4cf1-a944-042474eaea74 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:15.836728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c33c3-f70b-4488-9135-132ac2185630 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:15.837391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c33c3-f70b-4488-9135-132ac2185630 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.837715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c33c3-f70b-4488-9135-132ac2185630 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.837996 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5c33c3-f70b-4488-9135-132ac2185630 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.838724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c33c3-f70b-4488-9135-132ac2185630 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:15.840318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5c33c3-f70b-4488-9135-132ac2185630 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:15.855061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1059/2110] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:15.856161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c33c3-f70b-4488-9135-132ac2185630 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:15.868156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5c33c3-f70b-4488-9135-132ac2185630 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:15.884986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '05e0691b14804cc5b4fe120fc5dea871', 'request_id': 'req-2753a3a9-10f1-492b-ac12-7ce39b1faec6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.893087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '05e0691b14804cc5b4fe120fc5dea871', 'request_id': 'req-2753a3a9-10f1-492b-ac12-7ce39b1faec6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.899811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '05e0691b14804cc5b4fe120fc5dea871', 'request_id': 'req-2753a3a9-10f1-492b-ac12-7ce39b1faec6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.914515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] The token cache is being invalidate because project 05e0691b14804cc5b4fe120fc5dea871 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:15.914844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.915528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2753a3a9-10f1-492b-ac12-7ce39b1faec6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:15.920430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1052/2111] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:15 2022] DELETE /identity/v3/projects/05e0691b14804cc5b4fe120fc5dea871 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:15.925635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26813c1-518c-4975-a12e-f039928ec27b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.925926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26813c1-518c-4975-a12e-f039928ec27b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.926211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26813c1-518c-4975-a12e-f039928ec27b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.945103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c10a44ac-8340-4b59-9935-303ea8681061 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:15.945869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d26813c1-518c-4975-a12e-f039928ec27b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:15.946061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a44ac-8340-4b59-9935-303ea8681061 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.946319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a44ac-8340-4b59-9935-303ea8681061 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.946591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c10a44ac-8340-4b59-9935-303ea8681061 admin admin] PATH_INFO: `/v3/projects/d66a7bac21e44b72a59bb74c917daae1/users/67416d4eea6946fdb77a09242e58a12f/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.947107 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d26813c1-518c-4975-a12e-f039928ec27b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:15.956455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1053/2112] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:15.963683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a44ac-8340-4b59-9935-303ea8681061 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d66a7bac21e44b72a59bb74c917daae1, user_id=67416d4eea6946fdb77a09242e58a12f, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:15.965485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd34e1d-8b34-4d47-a797-9797e33dfd2b None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:15.965985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd34e1d-8b34-4d47-a797-9797e33dfd2b None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:15.966710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c10a44ac-8340-4b59-9935-303ea8681061 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:15.967018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd34e1d-8b34-4d47-a797-9797e33dfd2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:15.985569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1060/2113] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:15 2022] PUT /identity/v3/projects/d66a7bac21e44b72a59bb74c917daae1/users/67416d4eea6946fdb77a09242e58a12f/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:15.991539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4fd34e1d-8b34-4d47-a797-9797e33dfd2b None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:15.992764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4fd34e1d-8b34-4d47-a797-9797e33dfd2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:16.022793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:16.054205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef55495-97b4-42cc-b4d0-ca86cb225917 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:16.054603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef55495-97b4-42cc-b4d0-ca86cb225917 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:16.054881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef55495-97b4-42cc-b4d0-ca86cb225917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:16.063379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1054/2114] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:16.068104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd34e1d-8b34-4d47-a797-9797e33dfd2b None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:16.077502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd34e1d-8b34-4d47-a797-9797e33dfd2b None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:16.082173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8c37b-7c9d-420d-95d6-19440fc4103f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:16.082499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-aef55495-97b4-42cc-b4d0-ca86cb225917 None None] MFA Rules not processed for user `67416d4eea6946fdb77a09242e58a12f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:16.082813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8c37b-7c9d-420d-95d6-19440fc4103f admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:16.083853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8c37b-7c9d-420d-95d6-19440fc4103f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:16.084304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc8c37b-7c9d-420d-95d6-19440fc4103f admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:16.084528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aef55495-97b4-42cc-b4d0-ca86cb225917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:16.085043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8c37b-7c9d-420d-95d6-19440fc4103f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:16.085349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8c37b-7c9d-420d-95d6-19440fc4103f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:16.087069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc8c37b-7c9d-420d-95d6-19440fc4103f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:16.099815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1055/2115] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:25:16.104332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8c37b-7c9d-420d-95d6-19440fc4103f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:16.114577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc8c37b-7c9d-420d-95d6-19440fc4103f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:16.119441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c441271-c44e-4d67-b027-1722221ee82c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:16.120074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c441271-c44e-4d67-b027-1722221ee82c admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:16.120356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c441271-c44e-4d67-b027-1722221ee82c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:16.120631 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c441271-c44e-4d67-b027-1722221ee82c admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:16.121305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c441271-c44e-4d67-b027-1722221ee82c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:16.122853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:16.123391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c441271-c44e-4d67-b027-1722221ee82c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:16.151120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1056/2116] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:16 2022] POST /identity/v3/projects => generated 365 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:16.152498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1061/2117] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:16 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:16.155916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c441271-c44e-4d67-b027-1722221ee82c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:16.164585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef55495-97b4-42cc-b4d0-ca86cb225917 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:16.165997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aef55495-97b4-42cc-b4d0-ca86cb225917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:16.221301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c441271-c44e-4d67-b027-1722221ee82c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:16.227370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf450fff-4937-4fd1-9c3d-972ace0034d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:16.228239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf450fff-4937-4fd1-9c3d-972ace0034d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:16.228685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf450fff-4937-4fd1-9c3d-972ace0034d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:16.228968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf450fff-4937-4fd1-9c3d-972ace0034d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:16.229719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf450fff-4937-4fd1-9c3d-972ace0034d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:16.231689 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf450fff-4937-4fd1-9c3d-972ace0034d6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:16.232675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef55495-97b4-42cc-b4d0-ca86cb225917 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:16.255795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1057/2118] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:16 2022] POST /identity/v3/users => generated 283 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:25:16.258591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf450fff-4937-4fd1-9c3d-972ace0034d6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:16.268984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf450fff-4937-4fd1-9c3d-972ace0034d6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:16.275460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae9de1-d024-4e82-8d9b-8d6473d9d4a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:16.276211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae9de1-d024-4e82-8d9b-8d6473d9d4a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:16.276497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae9de1-d024-4e82-8d9b-8d6473d9d4a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:16.276784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aae9de1-d024-4e82-8d9b-8d6473d9d4a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:16.277542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae9de1-d024-4e82-8d9b-8d6473d9d4a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:16.279217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aae9de1-d024-4e82-8d9b-8d6473d9d4a4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:16.293687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1058/2119] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:16.295207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae9de1-d024-4e82-8d9b-8d6473d9d4a4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:16.297735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b53f394-7c4f-4bbe-b4de-4a0964358f1f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:16.298393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53f394-7c4f-4bbe-b4de-4a0964358f1f service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:16.298675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53f394-7c4f-4bbe-b4de-4a0964358f1f service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:16.298955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b53f394-7c4f-4bbe-b4de-4a0964358f1f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:16.306913 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aae9de1-d024-4e82-8d9b-8d6473d9d4a4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:16.319027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc155800-5c1f-4fb7-944f-7365275a473d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:16.320759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc155800-5c1f-4fb7-944f-7365275a473d admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:16.321057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc155800-5c1f-4fb7-944f-7365275a473d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:16.321334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc155800-5c1f-4fb7-944f-7365275a473d admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:16.322100 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc155800-5c1f-4fb7-944f-7365275a473d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:16.324783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc155800-5c1f-4fb7-944f-7365275a473d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:16.340150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1059/2120] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:16 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:16.344705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc155800-5c1f-4fb7-944f-7365275a473d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:16.361284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc155800-5c1f-4fb7-944f-7365275a473d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:16.365262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53f394-7c4f-4bbe-b4de-4a0964358f1f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:16.366826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b53f394-7c4f-4bbe-b4de-4a0964358f1f service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:16.367370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d59f30-ff46-41f1-b2fb-072041a2cb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:16.368282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d59f30-ff46-41f1-b2fb-072041a2cb44 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:16.368567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d59f30-ff46-41f1-b2fb-072041a2cb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:16.368867 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d59f30-ff46-41f1-b2fb-072041a2cb44 admin admin] PATH_INFO: `/v3/projects/97d243dd0d354280b89f156bed2bab76/users/e4fd259b12fe4d179a6fe922799a51d8/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:16.376084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1062/2121] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:16 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:25:16.384884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d59f30-ff46-41f1-b2fb-072041a2cb44 admin admin] RBAC: Authorizing `identity:create_grant(project_id=97d243dd0d354280b89f156bed2bab76, user_id=e4fd259b12fe4d179a6fe922799a51d8, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:16.397129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d59f30-ff46-41f1-b2fb-072041a2cb44 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:16.412179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1060/2122] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:16 2022] PUT /identity/v3/projects/97d243dd0d354280b89f156bed2bab76/users/e4fd259b12fe4d179a6fe922799a51d8/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:16.422494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9770e1-5e88-4e2f-a98d-419f147ecb36 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:16.422989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9770e1-5e88-4e2f-a98d-419f147ecb36 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:16.423102 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9770e1-5e88-4e2f-a98d-419f147ecb36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:16.453838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ce9770e1-5e88-4e2f-a98d-419f147ecb36 None None] MFA Rules not processed for user `e4fd259b12fe4d179a6fe922799a51d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:16.455051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ce9770e1-5e88-4e2f-a98d-419f147ecb36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:16.501895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:16.534172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1063/2123] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:16 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:16.685272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d59f30-ff46-41f1-b2fb-072041a2cb44 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:16.727985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:16.743985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d59f30-ff46-41f1-b2fb-072041a2cb44 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:16.748054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f90c067-9375-4d58-b790-d32c3a6377a7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:16.748705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c067-9375-4d58-b790-d32c3a6377a7 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:16.748990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c067-9375-4d58-b790-d32c3a6377a7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:16.749267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f90c067-9375-4d58-b790-d32c3a6377a7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:16.758292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90c067-9375-4d58-b790-d32c3a6377a7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:16.759647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f90c067-9375-4d58-b790-d32c3a6377a7 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:16.768593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1061/2124] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:16 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:22.309714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03083cf7-2fe7-4231-9b1a-a320cc069057 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:22.310292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03083cf7-2fe7-4231-9b1a-a320cc069057 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:22.310292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03083cf7-2fe7-4231-9b1a-a320cc069057 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:22.329921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-03083cf7-2fe7-4231-9b1a-a320cc069057 None None] MFA Rules not processed for user `67416d4eea6946fdb77a09242e58a12f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:22.331147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-03083cf7-2fe7-4231-9b1a-a320cc069057 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:22.391794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1064/2125] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:22 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:26.764462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=9 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:32.385166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=35 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:34.551387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b89499-f116-461d-a0b9-8394db80c576 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:34.551906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b89499-f116-461d-a0b9-8394db80c576 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:34.552004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9b89499-f116-461d-a0b9-8394db80c576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:34.573662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f9b89499-f116-461d-a0b9-8394db80c576 None None] MFA Rules not processed for user `afa79c7408524e979cf0e9a595fe731a`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:34.575777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f9b89499-f116-461d-a0b9-8394db80c576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:34.642973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1062/2126] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:34 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:44.636027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=25 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:46.210454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03083cf7-2fe7-4231-9b1a-a320cc069057 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:46.273962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03083cf7-2fe7-4231-9b1a-a320cc069057 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:46.279056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:46.279921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:46.280204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:46.280602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] PATH_INFO: `/v3/users/e4fd259b12fe4d179a6fe922799a51d8` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:46.281834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e4fd259b12fe4d179a6fe922799a51d8)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:46.283765 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:46.332107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e4fd259b12fe4d179a6fe922799a51d8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:46.343990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e4fd259b12fe4d179a6fe922799a51d8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:46.350910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e4fd259b12fe4d179a6fe922799a51d8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:46.351649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e4fd259b12fe4d179a6fe922799a51d8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:46.352302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e4fd259b12fe4d179a6fe922799a51d8'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:46.365848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1065/2127] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:46 2022] DELETE /identity/v3/users/e4fd259b12fe4d179a6fe922799a51d8 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:46.370927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e80ff-b5f2-498e-bb89-1a33361c3267 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:46.371400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e80ff-b5f2-498e-bb89-1a33361c3267 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:46.371489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1e80ff-b5f2-498e-bb89-1a33361c3267 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:46.389652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0e1e80ff-b5f2-498e-bb89-1a33361c3267 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:46.390942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0e1e80ff-b5f2-498e-bb89-1a33361c3267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:46.450903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1063/2128] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:46.460426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:46.461901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:46.521853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6a53e14-0592-4c95-9995-d2300b413ce4 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:46.526350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf079c18-bed1-475f-8e29-3e0439f3a31c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:46.527393 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079c18-bed1-475f-8e29-3e0439f3a31c service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:46.527677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079c18-bed1-475f-8e29-3e0439f3a31c service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:46.527951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf079c18-bed1-475f-8e29-3e0439f3a31c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:46.537559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf079c18-bed1-475f-8e29-3e0439f3a31c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:46.538998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf079c18-bed1-475f-8e29-3e0439f3a31c service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:46.550368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1066/2129] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:47.137719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e80ff-b5f2-498e-bb89-1a33361c3267 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:47.139401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0e1e80ff-b5f2-498e-bb89-1a33361c3267 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:47.158236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1e80ff-b5f2-498e-bb89-1a33361c3267 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:47.158236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:47.158678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.159193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.159541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] PATH_INFO: `/v3/projects/97d243dd0d354280b89f156bed2bab76` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.161104 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=97d243dd0d354280b89f156bed2bab76)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:47.163152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:47.224182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97d243dd0d354280b89f156bed2bab76', 'request_id': 'req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:47.230716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97d243dd0d354280b89f156bed2bab76', 'request_id': 'req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:47.236414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97d243dd0d354280b89f156bed2bab76', 'request_id': 'req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:47.248888 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] The token cache is being invalidate because project 97d243dd0d354280b89f156bed2bab76 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:47.249179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:47.249825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-f8b5bc1b-44f0-48db-b3ff-ef43a8166dfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:47.251389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1064/2130] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:47 2022] DELETE /identity/v3/projects/97d243dd0d354280b89f156bed2bab76 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:47.257169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d350-f832-47a8-b709-aa67cd51b6f9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.257671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d350-f832-47a8-b709-aa67cd51b6f9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.257761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9297d350-f832-47a8-b709-aa67cd51b6f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.277588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-9297d350-f832-47a8-b709-aa67cd51b6f9 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:47.278883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9297d350-f832-47a8-b709-aa67cd51b6f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:47.296597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:47.354497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1067/2131] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:47.363319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d548754-3163-40cc-8f43-4df9c269f7d1 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.363613 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d548754-3163-40cc-8f43-4df9c269f7d1 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.363893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d548754-3163-40cc-8f43-4df9c269f7d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.382497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1d548754-3163-40cc-8f43-4df9c269f7d1 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:47.383753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1d548754-3163-40cc-8f43-4df9c269f7d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:47.392192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1065/2132] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:47.396356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297d350-f832-47a8-b709-aa67cd51b6f9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:47.406628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9297d350-f832-47a8-b709-aa67cd51b6f9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:47.411145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac08ba-3b7d-4a70-9f3e-637d342048ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:47.411791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac08ba-3b7d-4a70-9f3e-637d342048ba admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.412072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac08ba-3b7d-4a70-9f3e-637d342048ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.412348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ac08ba-3b7d-4a70-9f3e-637d342048ba admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.413066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac08ba-3b7d-4a70-9f3e-637d342048ba admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:47.413368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac08ba-3b7d-4a70-9f3e-637d342048ba admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:47.414994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ac08ba-3b7d-4a70-9f3e-637d342048ba admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:47.426347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1068/2133] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:47.430646 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d548754-3163-40cc-8f43-4df9c269f7d1 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:47.439870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d548754-3163-40cc-8f43-4df9c269f7d1 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:47.444552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270b518-84c2-4bfb-b09e-12114a04bb85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:47.445198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270b518-84c2-4bfb-b09e-12114a04bb85 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.445479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270b518-84c2-4bfb-b09e-12114a04bb85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.445787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0270b518-84c2-4bfb-b09e-12114a04bb85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.446480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270b518-84c2-4bfb-b09e-12114a04bb85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:47.449030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0270b518-84c2-4bfb-b09e-12114a04bb85 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:47.475681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1066/2134] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:47 2022] POST /identity/v3/projects => generated 391 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:47.479752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac08ba-3b7d-4a70-9f3e-637d342048ba admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:47.490092 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ac08ba-3b7d-4a70-9f3e-637d342048ba admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:47.545060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f65cf-a5ec-48c3-a851-8b88e693cd05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:47.545760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f65cf-a5ec-48c3-a851-8b88e693cd05 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.546039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f65cf-a5ec-48c3-a851-8b88e693cd05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.546315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f65cf-a5ec-48c3-a851-8b88e693cd05 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.547035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f65cf-a5ec-48c3-a851-8b88e693cd05 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:47.548964 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-602f65cf-a5ec-48c3-a851-8b88e693cd05 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:47.569059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1069/2135] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:47 2022] POST /identity/v3/users => generated 296 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:47.573320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270b518-84c2-4bfb-b09e-12114a04bb85 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:47.585494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0270b518-84c2-4bfb-b09e-12114a04bb85 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:47.590564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a22be-e4f6-4329-bb70-c092ad3e6984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:47.591495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a22be-e4f6-4329-bb70-c092ad3e6984 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.591781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a22be-e4f6-4329-bb70-c092ad3e6984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.592059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a22be-e4f6-4329-bb70-c092ad3e6984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.592790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a22be-e4f6-4329-bb70-c092ad3e6984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:47.594426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a22be-e4f6-4329-bb70-c092ad3e6984 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:47.605011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1067/2136] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:47.609046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f65cf-a5ec-48c3-a851-8b88e693cd05 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:47.619225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f65cf-a5ec-48c3-a851-8b88e693cd05 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:47.624855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d086d52f-9f2b-456c-a07e-207e3f9eb14e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:47.625540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086d52f-9f2b-456c-a07e-207e3f9eb14e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.625840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086d52f-9f2b-456c-a07e-207e3f9eb14e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.626122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d086d52f-9f2b-456c-a07e-207e3f9eb14e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.626862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086d52f-9f2b-456c-a07e-207e3f9eb14e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:47.628473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d086d52f-9f2b-456c-a07e-207e3f9eb14e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:47.639287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1070/2137] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:47.643308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a22be-e4f6-4329-bb70-c092ad3e6984 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:47.653721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a22be-e4f6-4329-bb70-c092ad3e6984 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:47.658370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2f279-1332-4093-9bc2-04d1a1a762d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:47.659247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2f279-1332-4093-9bc2-04d1a1a762d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.659530 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2f279-1332-4093-9bc2-04d1a1a762d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.659812 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df2f279-1332-4093-9bc2-04d1a1a762d1 admin admin] PATH_INFO: `/v3/projects/8164087bcf974f7a9cf2fa253fb64e64/users/44f79300a4f9426886b08c407aa23703/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.674429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2f279-1332-4093-9bc2-04d1a1a762d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8164087bcf974f7a9cf2fa253fb64e64, user_id=44f79300a4f9426886b08c407aa23703, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:47.677097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df2f279-1332-4093-9bc2-04d1a1a762d1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:47.692080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1068/2138] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:47 2022] PUT /identity/v3/projects/8164087bcf974f7a9cf2fa253fb64e64/users/44f79300a4f9426886b08c407aa23703/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:47.704117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76360b14-1137-4e34-b226-72e4210ca98e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.704406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76360b14-1137-4e34-b226-72e4210ca98e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.704686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76360b14-1137-4e34-b226-72e4210ca98e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.732725 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-76360b14-1137-4e34-b226-72e4210ca98e None None] MFA Rules not processed for user `44f79300a4f9426886b08c407aa23703`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:47.733903 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-76360b14-1137-4e34-b226-72e4210ca98e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:47.797975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1071/2139] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:47 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:47.802608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2f279-1332-4093-9bc2-04d1a1a762d1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:47.824769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:47.867944 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df2f279-1332-4093-9bc2-04d1a1a762d1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:47.872644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6d6d1-4a3e-4835-a1b0-e4480552f95f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:47.873301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6d6d1-4a3e-4835-a1b0-e4480552f95f admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.873611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6d6d1-4a3e-4835-a1b0-e4480552f95f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.873888 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa6d6d1-4a3e-4835-a1b0-e4480552f95f admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.874604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6d6d1-4a3e-4835-a1b0-e4480552f95f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:47.876945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa6d6d1-4a3e-4835-a1b0-e4480552f95f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:47.905915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1069/2140] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:47 2022] POST /identity/v3/projects => generated 393 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:47.907060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76360b14-1137-4e34-b226-72e4210ca98e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:47.918427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76360b14-1137-4e34-b226-72e4210ca98e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:47.930620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:47.980000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3635-ab2e-4b1c-862d-fff911a8e2ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:47.980902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3635-ab2e-4b1c-862d-fff911a8e2ad admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:47.981201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3635-ab2e-4b1c-862d-fff911a8e2ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:47.981640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-123a3635-ab2e-4b1c-862d-fff911a8e2ad admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:47.982344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a3635-ab2e-4b1c-862d-fff911a8e2ad admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:47.985016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-123a3635-ab2e-4b1c-862d-fff911a8e2ad admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:48.012720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1072/2141] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:47 2022] POST /identity/v3/users => generated 303 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:48.014230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6d6d1-4a3e-4835-a1b0-e4480552f95f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:48.026289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa6d6d1-4a3e-4835-a1b0-e4480552f95f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:48.031746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0ebbf-170d-42d9-837c-a22969cf92b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:48.032600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0ebbf-170d-42d9-837c-a22969cf92b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:48.032958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0ebbf-170d-42d9-837c-a22969cf92b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:48.033305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50c0ebbf-170d-42d9-837c-a22969cf92b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:48.034150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0ebbf-170d-42d9-837c-a22969cf92b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:48.035904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50c0ebbf-170d-42d9-837c-a22969cf92b3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:48.050594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1070/2142] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:48.051373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3635-ab2e-4b1c-862d-fff911a8e2ad admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:48.063778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-123a3635-ab2e-4b1c-862d-fff911a8e2ad admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:48.069409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37ac9d-7317-4a7f-bbd4-8018f8ab7c4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:48.070332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37ac9d-7317-4a7f-bbd4-8018f8ab7c4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:48.070616 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37ac9d-7317-4a7f-bbd4-8018f8ab7c4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:48.070887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc37ac9d-7317-4a7f-bbd4-8018f8ab7c4e admin admin] PATH_INFO: `/v3/projects/00ad1aa77c05466fb8dd339cd5b68fe4/users/af8174bf55044592b79afbcb113e40e9/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:48.098239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc37ac9d-7317-4a7f-bbd4-8018f8ab7c4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=00ad1aa77c05466fb8dd339cd5b68fe4, user_id=af8174bf55044592b79afbcb113e40e9, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:48.100989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc37ac9d-7317-4a7f-bbd4-8018f8ab7c4e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:48.119897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1073/2143] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:48 2022] PUT /identity/v3/projects/00ad1aa77c05466fb8dd339cd5b68fe4/users/af8174bf55044592b79afbcb113e40e9/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:48.124618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0ebbf-170d-42d9-837c-a22969cf92b3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:48.139040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50c0ebbf-170d-42d9-837c-a22969cf92b3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:48.203746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aed0788a-9870-461c-8d40-844dbda921e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:48.204422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0788a-9870-461c-8d40-844dbda921e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:48.204704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0788a-9870-461c-8d40-844dbda921e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:48.204981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0788a-9870-461c-8d40-844dbda921e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:48.205731 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0788a-9870-461c-8d40-844dbda921e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:48.207314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aed0788a-9870-461c-8d40-844dbda921e9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:48.217987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1071/2144] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:48 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:48.222226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37ac9d-7317-4a7f-bbd4-8018f8ab7c4e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:48.233825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc37ac9d-7317-4a7f-bbd4-8018f8ab7c4e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:48.238087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b9604-4bc4-4aff-8ab0-6bc43041b6ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:48.238910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b9604-4bc4-4aff-8ab0-6bc43041b6ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:48.239200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b9604-4bc4-4aff-8ab0-6bc43041b6ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:48.239479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1b9604-4bc4-4aff-8ab0-6bc43041b6ff admin admin] PATH_INFO: `/v3/projects/00ad1aa77c05466fb8dd339cd5b68fe4/users/af8174bf55044592b79afbcb113e40e9/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:48.241563 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b9604-4bc4-4aff-8ab0-6bc43041b6ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=00ad1aa77c05466fb8dd339cd5b68fe4, user_id=af8174bf55044592b79afbcb113e40e9, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:48.244107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1b9604-4bc4-4aff-8ab0-6bc43041b6ff admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:48.255819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1074/2145] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:48 2022] PUT /identity/v3/projects/00ad1aa77c05466fb8dd339cd5b68fe4/users/af8174bf55044592b79afbcb113e40e9/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:48.266948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86df0fa-ebef-4f73-a1da-16d859acb285 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:48.267236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86df0fa-ebef-4f73-a1da-16d859acb285 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:48.267509 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86df0fa-ebef-4f73-a1da-16d859acb285 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:48.299213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e86df0fa-ebef-4f73-a1da-16d859acb285 None None] MFA Rules not processed for user `af8174bf55044592b79afbcb113e40e9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:48.300427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e86df0fa-ebef-4f73-a1da-16d859acb285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:48.384399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1072/2146] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:48 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:48.401098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b9604-4bc4-4aff-8ab0-6bc43041b6ff admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:48.402933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8e1b9604-4bc4-4aff-8ab0-6bc43041b6ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:48.470375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1b9604-4bc4-4aff-8ab0-6bc43041b6ff admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:48.470827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f7e52-8ce4-48ca-af02-75e9032d711c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:48.471529 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f7e52-8ce4-48ca-af02-75e9032d711c service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:48.471818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f7e52-8ce4-48ca-af02-75e9032d711c service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:48.472096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-829f7e52-8ce4-48ca-af02-75e9032d711c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:48.482096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f7e52-8ce4-48ca-af02-75e9032d711c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:48.483533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-829f7e52-8ce4-48ca-af02-75e9032d711c service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:48.493354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1075/2147] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:48 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:48.541820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86df0fa-ebef-4f73-a1da-16d859acb285 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:48.543846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e86df0fa-ebef-4f73-a1da-16d859acb285 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:48.555712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86df0fa-ebef-4f73-a1da-16d859acb285 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:48.560080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f1bf27-9b4c-4410-90a7-d46524f97f0f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:48.560714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1bf27-9b4c-4410-90a7-d46524f97f0f service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:48.560997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1bf27-9b4c-4410-90a7-d46524f97f0f service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:48.561273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f1bf27-9b4c-4410-90a7-d46524f97f0f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:48.573907 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:48.623161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1bf27-9b4c-4410-90a7-d46524f97f0f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:48.624593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f1bf27-9b4c-4410-90a7-d46524f97f0f service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:48.633720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1073/2148] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:48 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:49.160384 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f7e52-8ce4-48ca-af02-75e9032d711c service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:49.161813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-829f7e52-8ce4-48ca-af02-75e9032d711c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:49.230132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-829f7e52-8ce4-48ca-af02-75e9032d711c service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:49.235187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:49.236073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:49.236364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:49.236646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] PATH_INFO: `/v3/users/bcc382b27abe4d9dbb354ca54ae7aca6` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:49.238155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bcc382b27abe4d9dbb354ca54ae7aca6)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:49.240099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:49.269204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:49.303938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bcc382b27abe4d9dbb354ca54ae7aca6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:49.315733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bcc382b27abe4d9dbb354ca54ae7aca6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:49.322578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bcc382b27abe4d9dbb354ca54ae7aca6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:49.323294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bcc382b27abe4d9dbb354ca54ae7aca6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:49.324757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bcc382b27abe4d9dbb354ca54ae7aca6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:49.336977 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1076/2149] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:49 2022] DELETE /identity/v3/users/bcc382b27abe4d9dbb354ca54ae7aca6 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:49.342521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b759662-da4d-43a4-ab50-65017c242425 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:49.342814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b759662-da4d-43a4-ab50-65017c242425 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:49.343119 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b759662-da4d-43a4-ab50-65017c242425 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:49.360338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5b759662-da4d-43a4-ab50-65017c242425 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:49.361563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5b759662-da4d-43a4-ab50-65017c242425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:49.420746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1074/2150] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:49.430653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:49.432102 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:49.493706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bacd2c7-c57a-485e-9617-d9cc5b4d1c42 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:49.498245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c4421-9682-47ce-b17d-0b62d009f7b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:49.499024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c4421-9682-47ce-b17d-0b62d009f7b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:49.499374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c4421-9682-47ce-b17d-0b62d009f7b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:49.499723 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c34c4421-9682-47ce-b17d-0b62d009f7b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:49.509710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c4421-9682-47ce-b17d-0b62d009f7b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:49.511151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c34c4421-9682-47ce-b17d-0b62d009f7b2 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:49.522512 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1077/2151] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:25:49.815039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b759662-da4d-43a4-ab50-65017c242425 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:49.816531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5b759662-da4d-43a4-ab50-65017c242425 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:49.829413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b759662-da4d-43a4-ab50-65017c242425 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:49.834069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:49.834930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:49.835213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:49.835489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] PATH_INFO: `/v3/projects/1dfb0029d2004a3da6df55eb8fc7b50f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:49.836996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=1dfb0029d2004a3da6df55eb8fc7b50f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:49.839170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:49.888528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1dfb0029d2004a3da6df55eb8fc7b50f', 'request_id': 'req-dd7f9b00-d516-43ba-9be4-04e19257f2bb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:49.894902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1dfb0029d2004a3da6df55eb8fc7b50f', 'request_id': 'req-dd7f9b00-d516-43ba-9be4-04e19257f2bb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:49.900325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1dfb0029d2004a3da6df55eb8fc7b50f', 'request_id': 'req-dd7f9b00-d516-43ba-9be4-04e19257f2bb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:49.912987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] The token cache is being invalidate because project 1dfb0029d2004a3da6df55eb8fc7b50f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:49.913274 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:49.913915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:49.915452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1075/2152] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:49 2022] DELETE /identity/v3/projects/1dfb0029d2004a3da6df55eb8fc7b50f => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:49.920030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c4421-9682-47ce-b17d-0b62d009f7b2 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:49.930718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c34c4421-9682-47ce-b17d-0b62d009f7b2 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:49.993803 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:49.994675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:49.994961 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:49.995239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] PATH_INFO: `/v3/users/2fc635dea4e54dc5a7ecb8cac10d5884` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:49.997120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fc635dea4e54dc5a7ecb8cac10d5884)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:49.999076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:50.036497 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:50.046686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fc635dea4e54dc5a7ecb8cac10d5884'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:50.059221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fc635dea4e54dc5a7ecb8cac10d5884'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:50.065883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fc635dea4e54dc5a7ecb8cac10d5884'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:50.066558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fc635dea4e54dc5a7ecb8cac10d5884'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:50.067158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3c286691-2a3a-485c-a482-6fc2a173687a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fc635dea4e54dc5a7ecb8cac10d5884'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:50.079578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1078/2153] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:49 2022] DELETE /identity/v3/users/2fc635dea4e54dc5a7ecb8cac10d5884 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:25:50.369473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:50.371149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:50.386819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f9b00-d516-43ba-9be4-04e19257f2bb admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:50.426572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:50.462219 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:50.463628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:50.464048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:50.465100 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] PATH_INFO: `/v3/projects/f94c9f5dfda9483e9b2cb200a45e5958` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:50.466672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f94c9f5dfda9483e9b2cb200a45e5958)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:50.468678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:50.523015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f94c9f5dfda9483e9b2cb200a45e5958', 'request_id': 'req-e48cf261-ead6-4570-9151-5ac3ad627be1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:50.530578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f94c9f5dfda9483e9b2cb200a45e5958', 'request_id': 'req-e48cf261-ead6-4570-9151-5ac3ad627be1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:50.537124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f94c9f5dfda9483e9b2cb200a45e5958', 'request_id': 'req-e48cf261-ead6-4570-9151-5ac3ad627be1'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:50.556551 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] The token cache is being invalidate because project f94c9f5dfda9483e9b2cb200a45e5958 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:50.557077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:50.558109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e48cf261-ead6-4570-9151-5ac3ad627be1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:50.560372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1076/2154] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:50 2022] DELETE /identity/v3/projects/f94c9f5dfda9483e9b2cb200a45e5958 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:50.567579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da39e22-fedf-4018-ae3f-8fd8b96c7b67 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:50.568075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da39e22-fedf-4018-ae3f-8fd8b96c7b67 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:50.568181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da39e22-fedf-4018-ae3f-8fd8b96c7b67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:50.587032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1da39e22-fedf-4018-ae3f-8fd8b96c7b67 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:50.588783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1da39e22-fedf-4018-ae3f-8fd8b96c7b67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:50.657119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1079/2155] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:50.667119 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade69ff-2f5e-466c-97c1-72939e344546 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:50.667412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade69ff-2f5e-466c-97c1-72939e344546 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:50.668487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bade69ff-2f5e-466c-97c1-72939e344546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:50.693807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bade69ff-2f5e-466c-97c1-72939e344546 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:50.695037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bade69ff-2f5e-466c-97c1-72939e344546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:50.704906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1077/2156] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:50.708780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da39e22-fedf-4018-ae3f-8fd8b96c7b67 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:50.719304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da39e22-fedf-4018-ae3f-8fd8b96c7b67 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:50.723966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1caa68-3a39-423c-a055-4efcebf785ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:50.724632 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1caa68-3a39-423c-a055-4efcebf785ad admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:50.724925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1caa68-3a39-423c-a055-4efcebf785ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:50.725208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1caa68-3a39-423c-a055-4efcebf785ad admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:50.725982 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1caa68-3a39-423c-a055-4efcebf785ad admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:50.726295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1caa68-3a39-423c-a055-4efcebf785ad admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:50.727892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1caa68-3a39-423c-a055-4efcebf785ad admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:50.743902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1080/2157] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:50.745086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade69ff-2f5e-466c-97c1-72939e344546 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:50.755292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bade69ff-2f5e-466c-97c1-72939e344546 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:50.760114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75110a6-8e1a-4302-9d42-09bb465c5104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:50.760773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75110a6-8e1a-4302-9d42-09bb465c5104 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:50.761055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75110a6-8e1a-4302-9d42-09bb465c5104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:50.761341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f75110a6-8e1a-4302-9d42-09bb465c5104 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:50.762192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75110a6-8e1a-4302-9d42-09bb465c5104 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:50.764127 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f75110a6-8e1a-4302-9d42-09bb465c5104 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:50.799145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1078/2158] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:50 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:25:50.803162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1caa68-3a39-423c-a055-4efcebf785ad admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:50.813891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1caa68-3a39-423c-a055-4efcebf785ad admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:50.878553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb5b94-41f3-4b27-a1a1-b1b65bc0eb7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:50.879236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb5b94-41f3-4b27-a1a1-b1b65bc0eb7b admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:50.879523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb5b94-41f3-4b27-a1a1-b1b65bc0eb7b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:50.879806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bebb5b94-41f3-4b27-a1a1-b1b65bc0eb7b admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:50.880521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb5b94-41f3-4b27-a1a1-b1b65bc0eb7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:50.882564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bebb5b94-41f3-4b27-a1a1-b1b65bc0eb7b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:50.901778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1081/2159] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:50 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:50.906436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75110a6-8e1a-4302-9d42-09bb465c5104 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:50.917183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f75110a6-8e1a-4302-9d42-09bb465c5104 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:50.922572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072a179-fdca-484c-a278-b30cbf18ba73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:50.923335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072a179-fdca-484c-a278-b30cbf18ba73 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:50.923623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072a179-fdca-484c-a278-b30cbf18ba73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:50.923901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e072a179-fdca-484c-a278-b30cbf18ba73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:50.924629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072a179-fdca-484c-a278-b30cbf18ba73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:50.926299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e072a179-fdca-484c-a278-b30cbf18ba73 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:50.939768 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1079/2160] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:50.943530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb5b94-41f3-4b27-a1a1-b1b65bc0eb7b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:50.954706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bebb5b94-41f3-4b27-a1a1-b1b65bc0eb7b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:50.959778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4150290c-4947-4477-be55-dd029a077b91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:50.960508 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150290c-4947-4477-be55-dd029a077b91 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:50.960796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150290c-4947-4477-be55-dd029a077b91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:50.961077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4150290c-4947-4477-be55-dd029a077b91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:50.961911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150290c-4947-4477-be55-dd029a077b91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:50.963585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4150290c-4947-4477-be55-dd029a077b91 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:50.976276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1082/2161] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:25:50.984498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072a179-fdca-484c-a278-b30cbf18ba73 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:50.996718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e072a179-fdca-484c-a278-b30cbf18ba73 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:51.002497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fe131-f892-413b-8698-066a9cc191b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:51.003540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fe131-f892-413b-8698-066a9cc191b5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:51.003823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fe131-f892-413b-8698-066a9cc191b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:51.004093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204fe131-f892-413b-8698-066a9cc191b5 admin admin] PATH_INFO: `/v3/projects/c0ceb5aec8764fc4aa335a72a4276b9a/users/d496796ae3ed48109ba9c7454e6f7cc7/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:51.021490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204fe131-f892-413b-8698-066a9cc191b5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c0ceb5aec8764fc4aa335a72a4276b9a, user_id=d496796ae3ed48109ba9c7454e6f7cc7, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:51.024209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204fe131-f892-413b-8698-066a9cc191b5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:51.042184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1080/2162] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:50 2022] PUT /identity/v3/projects/c0ceb5aec8764fc4aa335a72a4276b9a/users/d496796ae3ed48109ba9c7454e6f7cc7/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:51.053184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590fa57-25dd-447f-8f69-75cec040c884 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:51.054062 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590fa57-25dd-447f-8f69-75cec040c884 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:51.054858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5590fa57-25dd-447f-8f69-75cec040c884 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:51.085839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-5590fa57-25dd-447f-8f69-75cec040c884 None None] MFA Rules not processed for user `d496796ae3ed48109ba9c7454e6f7cc7`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:51.087161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5590fa57-25dd-447f-8f69-75cec040c884 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:51.100333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:51.163998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1083/2163] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:51 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:51.175371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fe131-f892-413b-8698-066a9cc191b5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:51.177049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-204fe131-f892-413b-8698-066a9cc191b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:51.236399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204fe131-f892-413b-8698-066a9cc191b5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:51.240042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267f10d4-d8b3-432f-a19f-06932e32cce5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:51.240679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f10d4-d8b3-432f-a19f-06932e32cce5 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:51.240958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f10d4-d8b3-432f-a19f-06932e32cce5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:51.241233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267f10d4-d8b3-432f-a19f-06932e32cce5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:51.250467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f10d4-d8b3-432f-a19f-06932e32cce5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:51.251837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267f10d4-d8b3-432f-a19f-06932e32cce5 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:51.260982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1081/2164] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:51 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:51.281628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590fa57-25dd-447f-8f69-75cec040c884 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:51.351149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5590fa57-25dd-447f-8f69-75cec040c884 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:51.357636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:51.358632 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:51.358923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:51.359276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] PATH_INFO: `/v3/users/d496796ae3ed48109ba9c7454e6f7cc7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:51.360607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d496796ae3ed48109ba9c7454e6f7cc7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:51.362731 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:51.418952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd496796ae3ed48109ba9c7454e6f7cc7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:51.438028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd496796ae3ed48109ba9c7454e6f7cc7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:51.446216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd496796ae3ed48109ba9c7454e6f7cc7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:51.447056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd496796ae3ed48109ba9c7454e6f7cc7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:51.447926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd496796ae3ed48109ba9c7454e6f7cc7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:51.467359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1084/2165] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:51 2022] DELETE /identity/v3/users/d496796ae3ed48109ba9c7454e6f7cc7 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:51.471980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364462bf-9749-4acb-8fac-eb9719e94d69 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:51.472465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364462bf-9749-4acb-8fac-eb9719e94d69 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:51.472465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364462bf-9749-4acb-8fac-eb9719e94d69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:51.488048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:51.494909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-364462bf-9749-4acb-8fac-eb9719e94d69 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:51.497059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-364462bf-9749-4acb-8fac-eb9719e94d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:51.600537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1082/2166] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:51.613601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:51.615415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:51.665333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:51.709328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b25ea494-993f-48d1-8c5b-e6d9e4b36ae0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:51.713344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85108ec-102a-42b2-80e5-70d90dbaa105 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:51.714044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85108ec-102a-42b2-80e5-70d90dbaa105 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:51.715122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85108ec-102a-42b2-80e5-70d90dbaa105 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:51.715420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85108ec-102a-42b2-80e5-70d90dbaa105 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:51.726375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85108ec-102a-42b2-80e5-70d90dbaa105 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:51.727824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85108ec-102a-42b2-80e5-70d90dbaa105 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:51.752819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1085/2167] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:52.264200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364462bf-9749-4acb-8fac-eb9719e94d69 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:52.267038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-364462bf-9749-4acb-8fac-eb9719e94d69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:52.292979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364462bf-9749-4acb-8fac-eb9719e94d69 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:52.303221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:52.304296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:52.304639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:52.304929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] PATH_INFO: `/v3/projects/c0ceb5aec8764fc4aa335a72a4276b9a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:52.306771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] RBAC: Authorizing `identity:delete_project(project_id=c0ceb5aec8764fc4aa335a72a4276b9a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:52.309211 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:52.439059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c0ceb5aec8764fc4aa335a72a4276b9a', 'request_id': 'req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:52.445894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c0ceb5aec8764fc4aa335a72a4276b9a', 'request_id': 'req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:52.454030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c0ceb5aec8764fc4aa335a72a4276b9a', 'request_id': 'req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:52.470822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] The token cache is being invalidate because project c0ceb5aec8764fc4aa335a72a4276b9a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:52.471418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:52.472386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d826f375-61ff-4b97-8d7e-c8cc09dfb84b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:52.474676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1083/2168] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:52 2022] DELETE /identity/v3/projects/c0ceb5aec8764fc4aa335a72a4276b9a => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:52.479905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb8314-94aa-473b-8874-b33846d545dc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:52.480303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb8314-94aa-473b-8874-b33846d545dc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:52.480749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbeb8314-94aa-473b-8874-b33846d545dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:52.502439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-bbeb8314-94aa-473b-8874-b33846d545dc None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:52.503779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bbeb8314-94aa-473b-8874-b33846d545dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:52.568212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1086/2169] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:52.577423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3160a49-b93c-4cc8-abc9-5f8be3d80c20 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:52.577423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3160a49-b93c-4cc8-abc9-5f8be3d80c20 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:52.577772 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3160a49-b93c-4cc8-abc9-5f8be3d80c20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:52.598060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e3160a49-b93c-4cc8-abc9-5f8be3d80c20 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:52.599563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e3160a49-b93c-4cc8-abc9-5f8be3d80c20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:52.609196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1084/2170] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:52.614237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb8314-94aa-473b-8874-b33846d545dc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:52.625444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbeb8314-94aa-473b-8874-b33846d545dc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:52.639366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3b66c-b338-4888-b51d-e3eb09b537d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:52.640043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3b66c-b338-4888-b51d-e3eb09b537d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:52.640330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3b66c-b338-4888-b51d-e3eb09b537d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:52.640799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb3b66c-b338-4888-b51d-e3eb09b537d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:52.641740 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3b66c-b338-4888-b51d-e3eb09b537d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:52.642058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3b66c-b338-4888-b51d-e3eb09b537d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:52.643974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cb3b66c-b338-4888-b51d-e3eb09b537d4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:52.657745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1087/2171] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:52.662352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3160a49-b93c-4cc8-abc9-5f8be3d80c20 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:52.673782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3160a49-b93c-4cc8-abc9-5f8be3d80c20 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:52.678779 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a85731-7907-4384-a9a5-65aacb085def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:52.679525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a85731-7907-4384-a9a5-65aacb085def admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:52.679904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a85731-7907-4384-a9a5-65aacb085def admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:52.680257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4a85731-7907-4384-a9a5-65aacb085def admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:52.681023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a85731-7907-4384-a9a5-65aacb085def admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:52.683712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4a85731-7907-4384-a9a5-65aacb085def admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:52.721399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1085/2172] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:52 2022] POST /identity/v3/projects => generated 371 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:25:52.725591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3b66c-b338-4888-b51d-e3eb09b537d4 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:52.737834 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb3b66c-b338-4888-b51d-e3eb09b537d4 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:52.817127 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c05cf-fca1-4e97-9506-3bc74bdba2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:52.817912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c05cf-fca1-4e97-9506-3bc74bdba2bc admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:52.818193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c05cf-fca1-4e97-9506-3bc74bdba2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:52.818524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8c05cf-fca1-4e97-9506-3bc74bdba2bc admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:52.819156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c05cf-fca1-4e97-9506-3bc74bdba2bc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:52.821243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8c05cf-fca1-4e97-9506-3bc74bdba2bc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:52.847819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1088/2173] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:52 2022] POST /identity/v3/users => generated 286 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:52.856782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a85731-7907-4384-a9a5-65aacb085def admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:52.872400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4a85731-7907-4384-a9a5-65aacb085def admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:52.878778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169b99a-7e08-4f9c-b661-f09df9a5c727 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:52.879686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169b99a-7e08-4f9c-b661-f09df9a5c727 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:52.879947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169b99a-7e08-4f9c-b661-f09df9a5c727 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:52.880455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3169b99a-7e08-4f9c-b661-f09df9a5c727 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:52.881371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169b99a-7e08-4f9c-b661-f09df9a5c727 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:52.884205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3169b99a-7e08-4f9c-b661-f09df9a5c727 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:52.900310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1086/2174] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:52.904672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c05cf-fca1-4e97-9506-3bc74bdba2bc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:52.915611 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8c05cf-fca1-4e97-9506-3bc74bdba2bc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:52.921827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8fc495f-c8fd-4f95-84d2-bd8817e13ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:52.922881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc495f-c8fd-4f95-84d2-bd8817e13ec4 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:52.923292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc495f-c8fd-4f95-84d2-bd8817e13ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:52.923685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8fc495f-c8fd-4f95-84d2-bd8817e13ec4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:52.924656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc495f-c8fd-4f95-84d2-bd8817e13ec4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:52.926555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8fc495f-c8fd-4f95-84d2-bd8817e13ec4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:52.941554 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1089/2175] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:52 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:52.946396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169b99a-7e08-4f9c-b661-f09df9a5c727 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:52.966988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3169b99a-7e08-4f9c-b661-f09df9a5c727 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:52.977553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f7082-28f5-4aa3-80f1-923cd145a81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:52.979445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f7082-28f5-4aa3-80f1-923cd145a81c admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:52.980677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f7082-28f5-4aa3-80f1-923cd145a81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:52.981129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7f7082-28f5-4aa3-80f1-923cd145a81c admin admin] PATH_INFO: `/v3/projects/fc033f2529454d9fa2ea59bd062c4cc8/users/eb57bef8081a49d4bcb1bb7884cecb6f/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.007668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f7082-28f5-4aa3-80f1-923cd145a81c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fc033f2529454d9fa2ea59bd062c4cc8, user_id=eb57bef8081a49d4bcb1bb7884cecb6f, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:53.011216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7f7082-28f5-4aa3-80f1-923cd145a81c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:53.044794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1087/2176] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:52 2022] PUT /identity/v3/projects/fc033f2529454d9fa2ea59bd062c4cc8/users/eb57bef8081a49d4bcb1bb7884cecb6f/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:53.057092 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19a778-0991-4535-a02b-f695f633ae45 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:53.057424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19a778-0991-4535-a02b-f695f633ae45 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:53.057709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b19a778-0991-4535-a02b-f695f633ae45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.092541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-2b19a778-0991-4535-a02b-f695f633ae45 None None] MFA Rules not processed for user `eb57bef8081a49d4bcb1bb7884cecb6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:53.093905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2b19a778-0991-4535-a02b-f695f633ae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:53.129385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:53.190919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1090/2177] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:53.210634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f7082-28f5-4aa3-80f1-923cd145a81c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:53.213116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bf7f7082-28f5-4aa3-80f1-923cd145a81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:53.269403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:53.318521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7f7082-28f5-4aa3-80f1-923cd145a81c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:53.327297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5662d-b7a7-47da-a048-6b0e85518870 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:53.327833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5662d-b7a7-47da-a048-6b0e85518870 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:53.328303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5662d-b7a7-47da-a048-6b0e85518870 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:53.328488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff5662d-b7a7-47da-a048-6b0e85518870 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.345758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5662d-b7a7-47da-a048-6b0e85518870 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:53.351208 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff5662d-b7a7-47da-a048-6b0e85518870 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:53.367574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19a778-0991-4535-a02b-f695f633ae45 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:53.370664 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-2b19a778-0991-4535-a02b-f695f633ae45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:53.388289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1088/2178] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:53 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 178 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:53.429391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5662d-b7a7-47da-a048-6b0e85518870 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:53.510390 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b19a778-0991-4535-a02b-f695f633ae45 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:53.516957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:53.519176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:53.519727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:53.520278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] PATH_INFO: `/v3/users/44f79300a4f9426886b08c407aa23703` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.522536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=44f79300a4f9426886b08c407aa23703)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:53.524789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:53.563532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff5662d-b7a7-47da-a048-6b0e85518870 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:53.568681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:53.569824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:53.570150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:53.570392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] PATH_INFO: `/v3/users/eb57bef8081a49d4bcb1bb7884cecb6f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.571938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb57bef8081a49d4bcb1bb7884cecb6f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:53.574403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:53.615610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '44f79300a4f9426886b08c407aa23703'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:53.658639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '44f79300a4f9426886b08c407aa23703'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:53.666986 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '44f79300a4f9426886b08c407aa23703'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:53.668331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '44f79300a4f9426886b08c407aa23703'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:53.668975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-c6dd5971-cea1-4967-b6b9-0e2a5f4adb79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '44f79300a4f9426886b08c407aa23703'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:53.681631 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb57bef8081a49d4bcb1bb7884cecb6f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:53.704108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb57bef8081a49d4bcb1bb7884cecb6f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:53.715515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb57bef8081a49d4bcb1bb7884cecb6f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:53.716952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb57bef8081a49d4bcb1bb7884cecb6f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:53.717663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3d1c3abf-b94e-4441-b931-de02e2903286 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb57bef8081a49d4bcb1bb7884cecb6f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:53.727763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1091/2179] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:53 2022] DELETE /identity/v3/users/44f79300a4f9426886b08c407aa23703 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:53.744474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717a3e7-c836-48f7-8092-5bfc8c440f46 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:53.745263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717a3e7-c836-48f7-8092-5bfc8c440f46 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:53.745263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c717a3e7-c836-48f7-8092-5bfc8c440f46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.750547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1089/2180] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:53 2022] DELETE /identity/v3/users/eb57bef8081a49d4bcb1bb7884cecb6f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:53.754943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a44f6-d794-4cc0-a62d-55078290b2b5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:53.755230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a44f6-d794-4cc0-a62d-55078290b2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:53.755510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9a44f6-d794-4cc0-a62d-55078290b2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.770220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c717a3e7-c836-48f7-8092-5bfc8c440f46 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:53.771441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c717a3e7-c836-48f7-8092-5bfc8c440f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:53.778791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3a9a44f6-d794-4cc0-a62d-55078290b2b5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:53.780443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3a9a44f6-d794-4cc0-a62d-55078290b2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:53.833344 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:53.835591 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:53.859873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1090/2181] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:53.865540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1092/2182] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:53.866809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a44f6-d794-4cc0-a62d-55078290b2b5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:53.868485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3a9a44f6-d794-4cc0-a62d-55078290b2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:53.875510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717a3e7-c836-48f7-8092-5bfc8c440f46 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:53.878663 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c717a3e7-c836-48f7-8092-5bfc8c440f46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:53.938197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a9a44f6-d794-4cc0-a62d-55078290b2b5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:53.942015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8a926-a338-4b03-b9ff-3ed514741b70 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:53.942671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8a926-a338-4b03-b9ff-3ed514741b70 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:53.942952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8a926-a338-4b03-b9ff-3ed514741b70 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:53.943226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a8a926-a338-4b03-b9ff-3ed514741b70 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.947376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c717a3e7-c836-48f7-8092-5bfc8c440f46 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:53.952854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d76f5-2790-4d58-90f5-24257d231f0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:53.953375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8a926-a338-4b03-b9ff-3ed514741b70 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:53.953600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d76f5-2790-4d58-90f5-24257d231f0d service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:53.953989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d76f5-2790-4d58-90f5-24257d231f0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:53.954267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a65d76f5-2790-4d58-90f5-24257d231f0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:53.954848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a8a926-a338-4b03-b9ff-3ed514741b70 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:53.963885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d76f5-2790-4d58-90f5-24257d231f0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:53.965563 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a65d76f5-2790-4d58-90f5-24257d231f0d service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:53.966367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1091/2183] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:53.976041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1093/2184] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:54.577234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8a926-a338-4b03-b9ff-3ed514741b70 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:54.579583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-86a8a926-a338-4b03-b9ff-3ed514741b70 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:54.610309 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a8a926-a338-4b03-b9ff-3ed514741b70 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:54.619242 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:54.620217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:54.624337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:54.624956 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] PATH_INFO: `/v3/projects/fc033f2529454d9fa2ea59bd062c4cc8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:54.628381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fc033f2529454d9fa2ea59bd062c4cc8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:54.633760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:54.668303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d76f5-2790-4d58-90f5-24257d231f0d service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:54.670608 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a65d76f5-2790-4d58-90f5-24257d231f0d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:54.695012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a65d76f5-2790-4d58-90f5-24257d231f0d service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:54.703808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:54.704817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:54.705081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:54.705366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] PATH_INFO: `/v3/projects/8164087bcf974f7a9cf2fa253fb64e64` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:54.706748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] RBAC: Authorizing `identity:delete_project(project_id=8164087bcf974f7a9cf2fa253fb64e64)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:54.708706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:54.732921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fc033f2529454d9fa2ea59bd062c4cc8', 'request_id': 'req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:54.740919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fc033f2529454d9fa2ea59bd062c4cc8', 'request_id': 'req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:54.749307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fc033f2529454d9fa2ea59bd062c4cc8', 'request_id': 'req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:54.766962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] The token cache is being invalidate because project fc033f2529454d9fa2ea59bd062c4cc8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:54.767198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:54.768129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:54.782083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1092/2185] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:54 2022] DELETE /identity/v3/projects/fc033f2529454d9fa2ea59bd062c4cc8 => generated 0 bytes in 193 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:54.822648 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8164087bcf974f7a9cf2fa253fb64e64', 'request_id': 'req-76b7d70e-8a6a-4474-9fd8-4808114fbccb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:54.830402 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8164087bcf974f7a9cf2fa253fb64e64', 'request_id': 'req-76b7d70e-8a6a-4474-9fd8-4808114fbccb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:54.838855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8164087bcf974f7a9cf2fa253fb64e64', 'request_id': 'req-76b7d70e-8a6a-4474-9fd8-4808114fbccb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:54.853481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] The token cache is being invalidate because project 8164087bcf974f7a9cf2fa253fb64e64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:54.853888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:54.854540 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:54.860429 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1094/2186] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:54 2022] DELETE /identity/v3/projects/8164087bcf974f7a9cf2fa253fb64e64 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:54.861536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:54.863179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:54.973857 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2481ceb-ae95-4b80-9ab8-fe5f8e85ac48 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:54.997035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:55.057019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:55.058611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:55.058897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:55.059247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] PATH_INFO: `/v3/users/af8174bf55044592b79afbcb113e40e9` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:55.060719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] RBAC: Authorizing `identity:delete_user(user_id=af8174bf55044592b79afbcb113e40e9)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:55.063710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:55.118745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'af8174bf55044592b79afbcb113e40e9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:55.135031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'af8174bf55044592b79afbcb113e40e9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:55.142064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'af8174bf55044592b79afbcb113e40e9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:55.142803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'af8174bf55044592b79afbcb113e40e9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:55.143417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-416515b6-e332-49f0-82c5-b00df0fdf906 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'af8174bf55044592b79afbcb113e40e9'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:55.160880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1093/2187] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:25:54 2022] DELETE /identity/v3/users/af8174bf55044592b79afbcb113e40e9 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:55.490168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:55.491818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:55.505729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b7d70e-8a6a-4474-9fd8-4808114fbccb admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:55.569843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:55.570715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:55.570997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:55.571271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] PATH_INFO: `/v3/projects/00ad1aa77c05466fb8dd339cd5b68fe4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:55.572606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] RBAC: Authorizing `identity:delete_project(project_id=00ad1aa77c05466fb8dd339cd5b68fe4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:55.574802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:55.592694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:55.625053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '00ad1aa77c05466fb8dd339cd5b68fe4', 'request_id': 'req-25cd9b0c-6954-443e-8188-8f74bda9ef22'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:55.630470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '00ad1aa77c05466fb8dd339cd5b68fe4', 'request_id': 'req-25cd9b0c-6954-443e-8188-8f74bda9ef22'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:55.637635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '00ad1aa77c05466fb8dd339cd5b68fe4', 'request_id': 'req-25cd9b0c-6954-443e-8188-8f74bda9ef22'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:55.648635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] The token cache is being invalidate because project 00ad1aa77c05466fb8dd339cd5b68fe4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:25:55.648936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:55.649565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-25cd9b0c-6954-443e-8188-8f74bda9ef22 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:25:55.651126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1095/2188] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:25:55 2022] DELETE /identity/v3/projects/00ad1aa77c05466fb8dd339cd5b68fe4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:55.656662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a7a20-6c7f-48e7-8a79-c1c761b54915 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:55.657138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a7a20-6c7f-48e7-8a79-c1c761b54915 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:55.657239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a4a7a20-6c7f-48e7-8a79-c1c761b54915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:55.675671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3a4a7a20-6c7f-48e7-8a79-c1c761b54915 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:55.676891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3a4a7a20-6c7f-48e7-8a79-c1c761b54915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:55.736297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1094/2189] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:55.745180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81428d-1d64-4e19-a1a5-ed164f12a74f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:55.746222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81428d-1d64-4e19-a1a5-ed164f12a74f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:55.746352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e81428d-1d64-4e19-a1a5-ed164f12a74f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:55.764832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3e81428d-1d64-4e19-a1a5-ed164f12a74f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:55.766084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3e81428d-1d64-4e19-a1a5-ed164f12a74f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:55.774168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1096/2190] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:55.778539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a7a20-6c7f-48e7-8a79-c1c761b54915 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:55.788355 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a4a7a20-6c7f-48e7-8a79-c1c761b54915 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:55.793483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb8e35-fcc6-479a-99a3-cb8b7864589e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:55.794168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb8e35-fcc6-479a-99a3-cb8b7864589e admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:55.794454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb8e35-fcc6-479a-99a3-cb8b7864589e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:55.794734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43bb8e35-fcc6-479a-99a3-cb8b7864589e admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:55.795458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb8e35-fcc6-479a-99a3-cb8b7864589e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:25:55.795768 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb8e35-fcc6-479a-99a3-cb8b7864589e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:55.797365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43bb8e35-fcc6-479a-99a3-cb8b7864589e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:55.814020 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1095/2191] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:25:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:25:55.816407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81428d-1d64-4e19-a1a5-ed164f12a74f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:55.826593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e81428d-1d64-4e19-a1a5-ed164f12a74f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:55.832726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4975226-4784-43f6-81c4-85553e2bca90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:55.833681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4975226-4784-43f6-81c4-85553e2bca90 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:55.833971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4975226-4784-43f6-81c4-85553e2bca90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:55.834256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4975226-4784-43f6-81c4-85553e2bca90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:55.834962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4975226-4784-43f6-81c4-85553e2bca90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:55.836987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4975226-4784-43f6-81c4-85553e2bca90 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:55.862837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1097/2192] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:55 2022] POST /identity/v3/projects => generated 407 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:55.867129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb8e35-fcc6-479a-99a3-cb8b7864589e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:55.878154 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43bb8e35-fcc6-479a-99a3-cb8b7864589e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:55.904380 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:55.937428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92214d7-f948-4988-9424-e4a7613a6b9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:55.938158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92214d7-f948-4988-9424-e4a7613a6b9c admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:55.938425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92214d7-f948-4988-9424-e4a7613a6b9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:55.938705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92214d7-f948-4988-9424-e4a7613a6b9c admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:55.939408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92214d7-f948-4988-9424-e4a7613a6b9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:55.941335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92214d7-f948-4988-9424-e4a7613a6b9c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:55.961335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1096/2193] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:55 2022] POST /identity/v3/users => generated 304 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:55.966145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4975226-4784-43f6-81c4-85553e2bca90 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:55.976436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4975226-4784-43f6-81c4-85553e2bca90 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:55.981811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1bb7a-7f42-49fa-aac6-c3c14237a3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:55.982472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1bb7a-7f42-49fa-aac6-c3c14237a3ba admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:55.982760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1bb7a-7f42-49fa-aac6-c3c14237a3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:55.983043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a1bb7a-7f42-49fa-aac6-c3c14237a3ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:55.983784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1bb7a-7f42-49fa-aac6-c3c14237a3ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:55.985383 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45a1bb7a-7f42-49fa-aac6-c3c14237a3ba admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:55.996501 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1098/2194] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:56.000390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92214d7-f948-4988-9424-e4a7613a6b9c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:56.010565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92214d7-f948-4988-9424-e4a7613a6b9c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:56.015346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d38fb1-1514-4cd2-9ff7-acabe5c2c778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:56.016001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d38fb1-1514-4cd2-9ff7-acabe5c2c778 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.016356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d38fb1-1514-4cd2-9ff7-acabe5c2c778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.016672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d38fb1-1514-4cd2-9ff7-acabe5c2c778 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.017402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d38fb1-1514-4cd2-9ff7-acabe5c2c778 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:56.019034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d38fb1-1514-4cd2-9ff7-acabe5c2c778 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:56.029394 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1097/2195] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:56.033667 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1bb7a-7f42-49fa-aac6-c3c14237a3ba admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:56.044138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a1bb7a-7f42-49fa-aac6-c3c14237a3ba admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:56.049070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4725731-698f-4580-bbfa-bd1f166abf5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:56.049997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4725731-698f-4580-bbfa-bd1f166abf5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.050284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4725731-698f-4580-bbfa-bd1f166abf5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.050697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4725731-698f-4580-bbfa-bd1f166abf5c admin admin] PATH_INFO: `/v3/projects/e9f0dd8bfd3b4e14959172fe56c81483/users/bbb61229d27f408f866e486a565a494c/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.069226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4725731-698f-4580-bbfa-bd1f166abf5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9f0dd8bfd3b4e14959172fe56c81483, user_id=bbb61229d27f408f866e486a565a494c, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:56.071973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4725731-698f-4580-bbfa-bd1f166abf5c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:56.085894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1099/2196] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:56 2022] PUT /identity/v3/projects/e9f0dd8bfd3b4e14959172fe56c81483/users/bbb61229d27f408f866e486a565a494c/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:56.096525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f4939-5cab-4c03-901d-45a03f621ea4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.096815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f4939-5cab-4c03-901d-45a03f621ea4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.097095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee4f4939-5cab-4c03-901d-45a03f621ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.126304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ee4f4939-5cab-4c03-901d-45a03f621ea4 None None] MFA Rules not processed for user `bbb61229d27f408f866e486a565a494c`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:56.127540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ee4f4939-5cab-4c03-901d-45a03f621ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:56.194446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1098/2197] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:56 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:25:56.199607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4725731-698f-4580-bbfa-bd1f166abf5c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:56.266283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4725731-698f-4580-bbfa-bd1f166abf5c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:56.274988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e456fbb-778e-42a5-99ae-3fe5a37596d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:56.275655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e456fbb-778e-42a5-99ae-3fe5a37596d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.275936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e456fbb-778e-42a5-99ae-3fe5a37596d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.276327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e456fbb-778e-42a5-99ae-3fe5a37596d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.277844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e456fbb-778e-42a5-99ae-3fe5a37596d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:56.279789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e456fbb-778e-42a5-99ae-3fe5a37596d5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:56.309546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1100/2198] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:25:56 2022] POST /identity/v3/projects => generated 407 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:56.310866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f4939-5cab-4c03-901d-45a03f621ea4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:56.327842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee4f4939-5cab-4c03-901d-45a03f621ea4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:56.389241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2485eab-df0c-49e8-be56-06854202f29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:56.389947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2485eab-df0c-49e8-be56-06854202f29c admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.390230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2485eab-df0c-49e8-be56-06854202f29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.390502 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2485eab-df0c-49e8-be56-06854202f29c admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.394895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2485eab-df0c-49e8-be56-06854202f29c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:56.396968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2485eab-df0c-49e8-be56-06854202f29c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:56.422058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1099/2199] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:25:56 2022] POST /identity/v3/users => generated 310 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:25:56.423738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e456fbb-778e-42a5-99ae-3fe5a37596d5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:56.434578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e456fbb-778e-42a5-99ae-3fe5a37596d5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:56.439207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6b18e-b7ce-464a-b8bc-cced16b04698 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:56.439868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6b18e-b7ce-464a-b8bc-cced16b04698 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.440156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6b18e-b7ce-464a-b8bc-cced16b04698 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.440437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6b18e-b7ce-464a-b8bc-cced16b04698 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.441164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6b18e-b7ce-464a-b8bc-cced16b04698 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:56.442816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6b18e-b7ce-464a-b8bc-cced16b04698 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:56.453746 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1101/2200] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:56.457755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2485eab-df0c-49e8-be56-06854202f29c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:56.468783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2485eab-df0c-49e8-be56-06854202f29c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:56.473490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e81981-01bd-4899-ab6e-1a1646152d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:56.474399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e81981-01bd-4899-ab6e-1a1646152d51 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.474687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e81981-01bd-4899-ab6e-1a1646152d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.474970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e81981-01bd-4899-ab6e-1a1646152d51 admin admin] PATH_INFO: `/v3/projects/56fd5260ae154e9692a199299e61724b/users/01dcba0f75544430b5a87a194feba732/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.490523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e81981-01bd-4899-ab6e-1a1646152d51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56fd5260ae154e9692a199299e61724b, user_id=01dcba0f75544430b5a87a194feba732, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:56.493193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e81981-01bd-4899-ab6e-1a1646152d51 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:56.506514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1100/2201] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:56 2022] PUT /identity/v3/projects/56fd5260ae154e9692a199299e61724b/users/01dcba0f75544430b5a87a194feba732/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:56.510467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6b18e-b7ce-464a-b8bc-cced16b04698 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:56.521082 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6b18e-b7ce-464a-b8bc-cced16b04698 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:56.530765 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:56.580933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16002fc7-6c25-42af-abed-a11ce7c71d81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:56.581634 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16002fc7-6c25-42af-abed-a11ce7c71d81 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.581919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16002fc7-6c25-42af-abed-a11ce7c71d81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.582192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16002fc7-6c25-42af-abed-a11ce7c71d81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.582911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16002fc7-6c25-42af-abed-a11ce7c71d81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:56.584501 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16002fc7-6c25-42af-abed-a11ce7c71d81 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:56.599328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1102/2202] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:25:56 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:25:56.600578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e81981-01bd-4899-ab6e-1a1646152d51 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:56.607059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:25:56.613386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e81981-01bd-4899-ab6e-1a1646152d51 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:56.622884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc7545-8cd7-48dd-91cd-6e47098c62b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:56.624514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc7545-8cd7-48dd-91cd-6e47098c62b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.624514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc7545-8cd7-48dd-91cd-6e47098c62b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.624741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfc7545-8cd7-48dd-91cd-6e47098c62b6 admin admin] PATH_INFO: `/v3/projects/56fd5260ae154e9692a199299e61724b/users/01dcba0f75544430b5a87a194feba732/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.626773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc7545-8cd7-48dd-91cd-6e47098c62b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=56fd5260ae154e9692a199299e61724b, user_id=01dcba0f75544430b5a87a194feba732, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:56.629581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfc7545-8cd7-48dd-91cd-6e47098c62b6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:56.642053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1101/2203] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:25:56 2022] PUT /identity/v3/projects/56fd5260ae154e9692a199299e61724b/users/01dcba0f75544430b5a87a194feba732/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:25:56.653787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef59db-3cd2-45c1-953a-607386215856 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.654507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef59db-3cd2-45c1-953a-607386215856 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.654792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ef59db-3cd2-45c1-953a-607386215856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.693485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-27ef59db-3cd2-45c1-953a-607386215856 None None] MFA Rules not processed for user `01dcba0f75544430b5a87a194feba732`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:25:56.694732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-27ef59db-3cd2-45c1-953a-607386215856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:56.766887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1103/2204] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:25:56 2022] POST /identity/v3/auth/tokens => generated 3341 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:25:56.778278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc7545-8cd7-48dd-91cd-6e47098c62b6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:56.779992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7cfc7545-8cd7-48dd-91cd-6e47098c62b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:25:56.846449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfc7545-8cd7-48dd-91cd-6e47098c62b6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:56.850058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e14f1-5970-46a4-8f89-e308e68abffb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:56.850708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e14f1-5970-46a4-8f89-e308e68abffb service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:56.850995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e14f1-5970-46a4-8f89-e308e68abffb service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:56.851298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9e14f1-5970-46a4-8f89-e308e68abffb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:56.860838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e14f1-5970-46a4-8f89-e308e68abffb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:56.862292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9e14f1-5970-46a4-8f89-e308e68abffb service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:56.872417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1102/2205] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:56 2022] GET /identity/v3/auth/tokens => generated 3341 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:25:57.018343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef59db-3cd2-45c1-953a-607386215856 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:25:57.028134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ef59db-3cd2-45c1-953a-607386215856 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:25:57.031581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa8afb06-49c2-4db6-addb-541673f6ab26 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:25:57.032222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8afb06-49c2-4db6-addb-541673f6ab26 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:25:57.032510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8afb06-49c2-4db6-addb-541673f6ab26 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:25:57.032790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa8afb06-49c2-4db6-addb-541673f6ab26 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:25:57.089305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8afb06-49c2-4db6-addb-541673f6ab26 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:25:57.091081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa8afb06-49c2-4db6-addb-541673f6ab26 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:25:57.100098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1104/2206] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:25:57 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:06.867425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=30,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:07.095853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=85 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:12.474642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e14f1-5970-46a4-8f89-e308e68abffb service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:12.476071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-da9e14f1-5970-46a4-8f89-e308e68abffb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:12.547905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9e14f1-5970-46a4-8f89-e308e68abffb service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:12.553435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:12.554598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:12.554879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:12.555152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] PATH_INFO: `/v3/users/f8962313b5444eaebb2d5e5d982e0b29` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:12.556412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8962313b5444eaebb2d5e5d982e0b29)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:12.559666 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:12.614596 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8962313b5444eaebb2d5e5d982e0b29'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:12.628725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8962313b5444eaebb2d5e5d982e0b29'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:12.637228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8962313b5444eaebb2d5e5d982e0b29'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:12.638033 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8962313b5444eaebb2d5e5d982e0b29'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:12.638652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8962313b5444eaebb2d5e5d982e0b29'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:12.655428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1103/2207] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:12 2022] DELETE /identity/v3/users/f8962313b5444eaebb2d5e5d982e0b29 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:12.659894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf4a7d-76ef-4ac0-b5bc-e53a97eca08c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:12.660400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf4a7d-76ef-4ac0-b5bc-e53a97eca08c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:12.660489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcdf4a7d-76ef-4ac0-b5bc-e53a97eca08c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:12.694221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fcdf4a7d-76ef-4ac0-b5bc-e53a97eca08c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:12.696157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fcdf4a7d-76ef-4ac0-b5bc-e53a97eca08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:12.781434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1105/2208] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:12.791204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:12.793648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:12.867346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9646a12-2052-4ee8-8dd7-e33ddc5fb0ab admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:12.872453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82212f5-c6c0-4236-879c-baa674702ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:12.873244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82212f5-c6c0-4236-879c-baa674702ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:12.873622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82212f5-c6c0-4236-879c-baa674702ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:12.873961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82212f5-c6c0-4236-879c-baa674702ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:12.885894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82212f5-c6c0-4236-879c-baa674702ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:12.887654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82212f5-c6c0-4236-879c-baa674702ec8 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:12.898248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1104/2209] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:12.917610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf4a7d-76ef-4ac0-b5bc-e53a97eca08c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:12.918974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fcdf4a7d-76ef-4ac0-b5bc-e53a97eca08c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:12.930965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcdf4a7d-76ef-4ac0-b5bc-e53a97eca08c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:12.937182 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:12.938292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:12.939081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:12.940692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] PATH_INFO: `/v3/users/bbb61229d27f408f866e486a565a494c` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:12.942152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=bbb61229d27f408f866e486a565a494c)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:12.944555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.008181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bbb61229d27f408f866e486a565a494c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.023599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bbb61229d27f408f866e486a565a494c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.033043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bbb61229d27f408f866e486a565a494c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.033839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bbb61229d27f408f866e486a565a494c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.035189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bbb61229d27f408f866e486a565a494c'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.051060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1106/2210] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:12 2022] DELETE /identity/v3/users/bbb61229d27f408f866e486a565a494c => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:13.055812 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4621537-0119-4720-82ad-e5eeaf88a463 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.056066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4621537-0119-4720-82ad-e5eeaf88a463 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.056336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4621537-0119-4720-82ad-e5eeaf88a463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.079254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f4621537-0119-4720-82ad-e5eeaf88a463 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:13.080487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f4621537-0119-4720-82ad-e5eeaf88a463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:13.100894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:13.153419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1105/2211] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:13.163170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.164593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:13.225142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd809446-20fd-4ba4-85b6-85d1a85a42ff admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.228902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ec3374-84f9-4381-a2d0-d4449afc3d64 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.229621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3374-84f9-4381-a2d0-d4449afc3d64 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.229911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3374-84f9-4381-a2d0-d4449afc3d64 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.230194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ec3374-84f9-4381-a2d0-d4449afc3d64 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.239671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec3374-84f9-4381-a2d0-d4449afc3d64 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.241116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ec3374-84f9-4381-a2d0-d4449afc3d64 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.251342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1107/2212] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:13.311139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4621537-0119-4720-82ad-e5eeaf88a463 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.313444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f4621537-0119-4720-82ad-e5eeaf88a463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:13.339222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4621537-0119-4720-82ad-e5eeaf88a463 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.344493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.345365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.345691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.345971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] PATH_INFO: `/v3/projects/12297d403c7f4d2cbd9b2e5bbc570128` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.347188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12297d403c7f4d2cbd9b2e5bbc570128)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.349114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.398848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12297d403c7f4d2cbd9b2e5bbc570128', 'request_id': 'req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.406047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12297d403c7f4d2cbd9b2e5bbc570128', 'request_id': 'req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.411881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12297d403c7f4d2cbd9b2e5bbc570128', 'request_id': 'req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.425365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] The token cache is being invalidate because project 12297d403c7f4d2cbd9b2e5bbc570128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:13.425832 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.426768 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6fa8b9e2-44b3-48c5-91de-42f2b01c62d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.428597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1106/2213] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:13 2022] DELETE /identity/v3/projects/12297d403c7f4d2cbd9b2e5bbc570128 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:13.434696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97358e-4d57-4438-a53b-21e426172e4f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.434994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97358e-4d57-4438-a53b-21e426172e4f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.435274 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba97358e-4d57-4438-a53b-21e426172e4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.453369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ba97358e-4d57-4438-a53b-21e426172e4f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:13.454623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ba97358e-4d57-4438-a53b-21e426172e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:13.464341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:13.519373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1108/2214] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:13.526875 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5bb4-7171-4402-a9e5-195ccc93f779 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.527414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5bb4-7171-4402-a9e5-195ccc93f779 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.527414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d2e5bb4-7171-4402-a9e5-195ccc93f779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.545990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7d2e5bb4-7171-4402-a9e5-195ccc93f779 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:13.547267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7d2e5bb4-7171-4402-a9e5-195ccc93f779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:13.555910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1107/2215] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:13.560491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97358e-4d57-4438-a53b-21e426172e4f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.561011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e5bb4-7171-4402-a9e5-195ccc93f779 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.562039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ba97358e-4d57-4438-a53b-21e426172e4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:13.570255 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d2e5bb4-7171-4402-a9e5-195ccc93f779 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.576016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de447559-30e5-4ac4-ba1e-403ed46a7bd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.576681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de447559-30e5-4ac4-ba1e-403ed46a7bd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.576959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de447559-30e5-4ac4-ba1e-403ed46a7bd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.577232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de447559-30e5-4ac4-ba1e-403ed46a7bd4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.577993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de447559-30e5-4ac4-ba1e-403ed46a7bd4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:26:13.578306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de447559-30e5-4ac4-ba1e-403ed46a7bd4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.579083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba97358e-4d57-4438-a53b-21e426172e4f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.580230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de447559-30e5-4ac4-ba1e-403ed46a7bd4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.584024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.584956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.585242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.585538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] PATH_INFO: `/v3/projects/e9f0dd8bfd3b4e14959172fe56c81483` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.587262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9f0dd8bfd3b4e14959172fe56c81483)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.589431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.594788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1108/2216] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:26:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:26:13.599116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de447559-30e5-4ac4-ba1e-403ed46a7bd4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.608936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de447559-30e5-4ac4-ba1e-403ed46a7bd4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.613485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257bcec-0717-45b1-92b1-41122e7d253b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.614173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257bcec-0717-45b1-92b1-41122e7d253b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.614458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257bcec-0717-45b1-92b1-41122e7d253b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.614739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7257bcec-0717-45b1-92b1-41122e7d253b admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.615430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257bcec-0717-45b1-92b1-41122e7d253b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.617318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7257bcec-0717-45b1-92b1-41122e7d253b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.643135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9f0dd8bfd3b4e14959172fe56c81483', 'request_id': 'req-1f2245bb-63c7-42f4-9aad-da9638346608'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.645797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1109/2217] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:13 2022] POST /identity/v3/projects => generated 395 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:26:13.649062 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9f0dd8bfd3b4e14959172fe56c81483', 'request_id': 'req-1f2245bb-63c7-42f4-9aad-da9638346608'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.649908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257bcec-0717-45b1-92b1-41122e7d253b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.657866 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9f0dd8bfd3b4e14959172fe56c81483', 'request_id': 'req-1f2245bb-63c7-42f4-9aad-da9638346608'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.669190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] The token cache is being invalidate because project e9f0dd8bfd3b4e14959172fe56c81483 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:13.669809 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.670143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.671709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1109/2218] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:13 2022] DELETE /identity/v3/projects/e9f0dd8bfd3b4e14959172fe56c81483 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:13.675577 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.676888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:13.717386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7257bcec-0717-45b1-92b1-41122e7d253b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.722693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b934c-9da7-4751-809c-44d3d6149194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.723377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b934c-9da7-4751-809c-44d3d6149194 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.723644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b934c-9da7-4751-809c-44d3d6149194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.723917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b934c-9da7-4751-809c-44d3d6149194 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.724600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b934c-9da7-4751-809c-44d3d6149194 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.726562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b934c-9da7-4751-809c-44d3d6149194 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.735354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:13.752771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1110/2219] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:13 2022] POST /identity/v3/users => generated 298 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:26:13.756570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b934c-9da7-4751-809c-44d3d6149194 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.756706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f2245bb-63c7-42f4-9aad-da9638346608 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.758439 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-da0b934c-9da7-4751-809c-44d3d6149194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:13.762669 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.763552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.763829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.764099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] PATH_INFO: `/v3/users/01dcba0f75544430b5a87a194feba732` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.766295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] RBAC: Authorizing `identity:delete_user(user_id=01dcba0f75544430b5a87a194feba732)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.768254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.772505 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b934c-9da7-4751-809c-44d3d6149194 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.778471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222af834-0288-462e-95c8-2f99a9e84d3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.779155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222af834-0288-462e-95c8-2f99a9e84d3c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.779444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222af834-0288-462e-95c8-2f99a9e84d3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.779724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222af834-0288-462e-95c8-2f99a9e84d3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.780464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222af834-0288-462e-95c8-2f99a9e84d3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.782571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222af834-0288-462e-95c8-2f99a9e84d3c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.797559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1111/2220] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:26:13.801625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222af834-0288-462e-95c8-2f99a9e84d3c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.813173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222af834-0288-462e-95c8-2f99a9e84d3c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.818961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f4fd7-7c30-4c55-84c3-3cff091bb516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.819656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f4fd7-7c30-4c55-84c3-3cff091bb516 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.820000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f4fd7-7c30-4c55-84c3-3cff091bb516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.820214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030f4fd7-7c30-4c55-84c3-3cff091bb516 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.820598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:13.821116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f4fd7-7c30-4c55-84c3-3cff091bb516 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.822762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030f4fd7-7c30-4c55-84c3-3cff091bb516 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.828278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '01dcba0f75544430b5a87a194feba732'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.840061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1112/2221] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:26:13.844420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f4fd7-7c30-4c55-84c3-3cff091bb516 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:13.852965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '01dcba0f75544430b5a87a194feba732'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.857992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030f4fd7-7c30-4c55-84c3-3cff091bb516 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:13.863844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40bbf6-da71-4e37-af66-274143cabea1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:13.864136 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '01dcba0f75544430b5a87a194feba732'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.864752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40bbf6-da71-4e37-af66-274143cabea1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.865024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40bbf6-da71-4e37-af66-274143cabea1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.865301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca40bbf6-da71-4e37-af66-274143cabea1 admin admin] PATH_INFO: `/v3/projects/ed168e10a8f54c5dab88fb1b5fc48c8c/users/ea26a71427d14ab08a84f2710d9ad0bc/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.865436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '01dcba0f75544430b5a87a194feba732'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.866212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-12edfa7d-e6d3-4434-98dc-d841efa19272 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '01dcba0f75544430b5a87a194feba732'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:13.884306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40bbf6-da71-4e37-af66-274143cabea1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed168e10a8f54c5dab88fb1b5fc48c8c, user_id=ea26a71427d14ab08a84f2710d9ad0bc, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:13.888149 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca40bbf6-da71-4e37-af66-274143cabea1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:13.891110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1110/2222] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:13 2022] DELETE /identity/v3/users/01dcba0f75544430b5a87a194feba732 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:26:13.917797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1113/2223] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:13 2022] PUT /identity/v3/projects/ed168e10a8f54c5dab88fb1b5fc48c8c/users/ea26a71427d14ab08a84f2710d9ad0bc/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:26:13.927496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1583407-313b-4934-b097-8ed3b522da3f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:13.927795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1583407-313b-4934-b097-8ed3b522da3f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:13.928072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1583407-313b-4934-b097-8ed3b522da3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:13.973958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b1583407-313b-4934-b097-8ed3b522da3f None None] MFA Rules not processed for user `ea26a71427d14ab08a84f2710d9ad0bc`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:13.975751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b1583407-313b-4934-b097-8ed3b522da3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:14.068788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1111/2224] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:13 2022] POST /identity/v3/auth/tokens => generated 3262 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:14.082339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40bbf6-da71-4e37-af66-274143cabea1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:14.087229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ca40bbf6-da71-4e37-af66-274143cabea1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:14.148320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca40bbf6-da71-4e37-af66-274143cabea1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:14.152744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e12af19-ed22-4327-a50e-684da1a31aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:14.153396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12af19-ed22-4327-a50e-684da1a31aec service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.153711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12af19-ed22-4327-a50e-684da1a31aec service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.153987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e12af19-ed22-4327-a50e-684da1a31aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.169290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12af19-ed22-4327-a50e-684da1a31aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:14.172034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e12af19-ed22-4327-a50e-684da1a31aec service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:14.185839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1114/2225] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:14 2022] GET /identity/v3/auth/tokens => generated 3262 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:14.229901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1583407-313b-4934-b097-8ed3b522da3f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:14.231438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b1583407-313b-4934-b097-8ed3b522da3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:14.357888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1583407-313b-4934-b097-8ed3b522da3f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:14.363181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:14.364052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.364336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.364612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] PATH_INFO: `/v3/projects/56fd5260ae154e9692a199299e61724b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.366261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=56fd5260ae154e9692a199299e61724b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:14.368207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:14.436778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '56fd5260ae154e9692a199299e61724b', 'request_id': 'req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.438135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:14.444133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '56fd5260ae154e9692a199299e61724b', 'request_id': 'req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.456258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '56fd5260ae154e9692a199299e61724b', 'request_id': 'req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.471537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] The token cache is being invalidate because project 56fd5260ae154e9692a199299e61724b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:14.471813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.472749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-576e35cf-f31b-4f5c-a21f-3fe4652e69d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:14.474918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1112/2226] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:14 2022] DELETE /identity/v3/projects/56fd5260ae154e9692a199299e61724b => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:14.480080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bad9c6-df61-4fdd-9524-938cee37c39c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.480434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bad9c6-df61-4fdd-9524-938cee37c39c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.480656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8bad9c6-df61-4fdd-9524-938cee37c39c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.504720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e8bad9c6-df61-4fdd-9524-938cee37c39c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:14.505957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e8bad9c6-df61-4fdd-9524-938cee37c39c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:14.573881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1115/2227] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:14.581535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7eb5eb-819e-4cf8-a2c3-b352bd998216 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.581837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7eb5eb-819e-4cf8-a2c3-b352bd998216 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.582118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7eb5eb-819e-4cf8-a2c3-b352bd998216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.602000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3d7eb5eb-819e-4cf8-a2c3-b352bd998216 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:14.603225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3d7eb5eb-819e-4cf8-a2c3-b352bd998216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:14.613041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1113/2228] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:14.617629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bad9c6-df61-4fdd-9524-938cee37c39c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:14.637539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8bad9c6-df61-4fdd-9524-938cee37c39c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:14.637539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34dd07-c155-4f0e-af11-277abe0f19d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:14.637539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34dd07-c155-4f0e-af11-277abe0f19d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.637539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34dd07-c155-4f0e-af11-277abe0f19d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.637886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec34dd07-c155-4f0e-af11-277abe0f19d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.637886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34dd07-c155-4f0e-af11-277abe0f19d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:26:14.637886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34dd07-c155-4f0e-af11-277abe0f19d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:14.638597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec34dd07-c155-4f0e-af11-277abe0f19d6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:14.657563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1116/2229] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:26:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:26:14.659651 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7eb5eb-819e-4cf8-a2c3-b352bd998216 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:14.674680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7eb5eb-819e-4cf8-a2c3-b352bd998216 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:14.677893 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cfb23-353f-4b0d-a01e-a25de20dfbcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:14.678818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cfb23-353f-4b0d-a01e-a25de20dfbcf admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.679342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cfb23-353f-4b0d-a01e-a25de20dfbcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.679729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1cfb23-353f-4b0d-a01e-a25de20dfbcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.680449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cfb23-353f-4b0d-a01e-a25de20dfbcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:14.683737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1cfb23-353f-4b0d-a01e-a25de20dfbcf admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:14.713159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1114/2230] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:14 2022] POST /identity/v3/projects => generated 385 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:26:14.718356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34dd07-c155-4f0e-af11-277abe0f19d6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:14.729730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec34dd07-c155-4f0e-af11-277abe0f19d6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:14.764425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:14.788569 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677548d-8372-47d3-af97-ab81153833ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:14.789585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677548d-8372-47d3-af97-ab81153833ae admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.789724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677548d-8372-47d3-af97-ab81153833ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.789972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e677548d-8372-47d3-af97-ab81153833ae admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.790668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677548d-8372-47d3-af97-ab81153833ae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:14.793305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e677548d-8372-47d3-af97-ab81153833ae admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:14.816326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1117/2231] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:14 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:14.820547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cfb23-353f-4b0d-a01e-a25de20dfbcf admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:14.831440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1cfb23-353f-4b0d-a01e-a25de20dfbcf admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:14.836492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8ce5f-9b98-4e9b-bdb9-ce271f7627e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:14.837188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8ce5f-9b98-4e9b-bdb9-ce271f7627e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.837473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8ce5f-9b98-4e9b-bdb9-ce271f7627e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.837788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a8ce5f-9b98-4e9b-bdb9-ce271f7627e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.838639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8ce5f-9b98-4e9b-bdb9-ce271f7627e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:14.840255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a8ce5f-9b98-4e9b-bdb9-ce271f7627e8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:14.851336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1115/2232] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:26:14.856552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677548d-8372-47d3-af97-ab81153833ae admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:14.866481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e677548d-8372-47d3-af97-ab81153833ae admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:14.871876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34a081d-50ea-49e9-9220-2a69b98100bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:14.872540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a081d-50ea-49e9-9220-2a69b98100bc admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.872824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a081d-50ea-49e9-9220-2a69b98100bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.873100 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34a081d-50ea-49e9-9220-2a69b98100bc admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.873864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a081d-50ea-49e9-9220-2a69b98100bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:14.875456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34a081d-50ea-49e9-9220-2a69b98100bc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:14.889342 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1118/2233] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:14.890492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8ce5f-9b98-4e9b-bdb9-ce271f7627e8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:14.900850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a8ce5f-9b98-4e9b-bdb9-ce271f7627e8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:14.906226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141db5db-60a6-4332-8d7f-b34fcf183bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:14.907101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141db5db-60a6-4332-8d7f-b34fcf183bc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.907385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141db5db-60a6-4332-8d7f-b34fcf183bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.907658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-141db5db-60a6-4332-8d7f-b34fcf183bc0 admin admin] PATH_INFO: `/v3/projects/76df65ce9eeb45f991d54aff5d39e203/users/47abd0e4b2b447b6bc44329cbf4347a2/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.923458 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141db5db-60a6-4332-8d7f-b34fcf183bc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=76df65ce9eeb45f991d54aff5d39e203, user_id=47abd0e4b2b447b6bc44329cbf4347a2, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:14.926276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-141db5db-60a6-4332-8d7f-b34fcf183bc0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:14.940267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1116/2234] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:14 2022] PUT /identity/v3/projects/76df65ce9eeb45f991d54aff5d39e203/users/47abd0e4b2b447b6bc44329cbf4347a2/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:14.951295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf6e6e-8082-4ffa-a589-4f78d9e1de46 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:14.951595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf6e6e-8082-4ffa-a589-4f78d9e1de46 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:14.951878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debf6e6e-8082-4ffa-a589-4f78d9e1de46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:14.978747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-debf6e6e-8082-4ffa-a589-4f78d9e1de46 None None] MFA Rules not processed for user `47abd0e4b2b447b6bc44329cbf4347a2`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:14.980243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-debf6e6e-8082-4ffa-a589-4f78d9e1de46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:15.050015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1119/2235] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:14 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:15.062369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141db5db-60a6-4332-8d7f-b34fcf183bc0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:15.064110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-141db5db-60a6-4332-8d7f-b34fcf183bc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:15.122188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-141db5db-60a6-4332-8d7f-b34fcf183bc0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:15.125994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96f07054-ed96-401d-963f-97859cb2826c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:15.126654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f07054-ed96-401d-963f-97859cb2826c service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.126940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f07054-ed96-401d-963f-97859cb2826c service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.127218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f07054-ed96-401d-963f-97859cb2826c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.136428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f07054-ed96-401d-963f-97859cb2826c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:15.137928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96f07054-ed96-401d-963f-97859cb2826c service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:15.147022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1117/2236] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:15 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:15.166475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf6e6e-8082-4ffa-a589-4f78d9e1de46 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:15.233866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debf6e6e-8082-4ffa-a589-4f78d9e1de46 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:15.238436 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:15.239310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.239594 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.239873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] PATH_INFO: `/v3/users/47abd0e4b2b447b6bc44329cbf4347a2` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.241106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=47abd0e4b2b447b6bc44329cbf4347a2)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:15.243065 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:15.289082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '47abd0e4b2b447b6bc44329cbf4347a2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.290222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:15.304144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '47abd0e4b2b447b6bc44329cbf4347a2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.311853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '47abd0e4b2b447b6bc44329cbf4347a2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.312566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '47abd0e4b2b447b6bc44329cbf4347a2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.313264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '47abd0e4b2b447b6bc44329cbf4347a2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:15.328463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1120/2237] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:15 2022] DELETE /identity/v3/users/47abd0e4b2b447b6bc44329cbf4347a2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:15.333077 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994fe4d-54f9-4f71-a807-c7b9ba5f8fa5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.333365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994fe4d-54f9-4f71-a807-c7b9ba5f8fa5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.333674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e994fe4d-54f9-4f71-a807-c7b9ba5f8fa5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.350886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e994fe4d-54f9-4f71-a807-c7b9ba5f8fa5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:15.352069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e994fe4d-54f9-4f71-a807-c7b9ba5f8fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:15.409459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1118/2238] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:15.420401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:15.421870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:15.482213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2fe9f6-0168-42a9-8247-c5188d0409d8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:15.486160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440e1ae7-92dc-45cf-907e-ebd1813bfc4f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:15.486759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e1ae7-92dc-45cf-907e-ebd1813bfc4f service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:15.487049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e1ae7-92dc-45cf-907e-ebd1813bfc4f service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:15.487330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440e1ae7-92dc-45cf-907e-ebd1813bfc4f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:15.497713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e1ae7-92dc-45cf-907e-ebd1813bfc4f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:15.499158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440e1ae7-92dc-45cf-907e-ebd1813bfc4f service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:15.514489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1121/2239] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:16.110549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994fe4d-54f9-4f71-a807-c7b9ba5f8fa5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.111997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e994fe4d-54f9-4f71-a807-c7b9ba5f8fa5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.127952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e994fe4d-54f9-4f71-a807-c7b9ba5f8fa5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.132629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.133487 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.133796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.134070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] PATH_INFO: `/v3/projects/76df65ce9eeb45f991d54aff5d39e203` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.135922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] RBAC: Authorizing `identity:delete_project(project_id=76df65ce9eeb45f991d54aff5d39e203)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.137932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.149409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:16.186706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '76df65ce9eeb45f991d54aff5d39e203', 'request_id': 'req-1d7f896f-a644-464d-8f40-4eac6cfcf062'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:16.192110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '76df65ce9eeb45f991d54aff5d39e203', 'request_id': 'req-1d7f896f-a644-464d-8f40-4eac6cfcf062'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:16.200058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '76df65ce9eeb45f991d54aff5d39e203', 'request_id': 'req-1d7f896f-a644-464d-8f40-4eac6cfcf062'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:16.211076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] The token cache is being invalidate because project 76df65ce9eeb45f991d54aff5d39e203 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:16.211504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:16.212132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1d7f896f-a644-464d-8f40-4eac6cfcf062 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:16.214459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1119/2240] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:16 2022] DELETE /identity/v3/projects/76df65ce9eeb45f991d54aff5d39e203 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:16.219618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa01d-920c-4e3b-baaf-edb06ceaddbd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.220077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa01d-920c-4e3b-baaf-edb06ceaddbd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.220257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3aa01d-920c-4e3b-baaf-edb06ceaddbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.238781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ff3aa01d-920c-4e3b-baaf-edb06ceaddbd None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:16.240004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ff3aa01d-920c-4e3b-baaf-edb06ceaddbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.302115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1122/2241] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:16.310406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f00972-e73d-4884-adb0-a4280e79f7db None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.310781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f00972-e73d-4884-adb0-a4280e79f7db None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.311131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f00972-e73d-4884-adb0-a4280e79f7db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.333021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-18f00972-e73d-4884-adb0-a4280e79f7db None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:16.334521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-18f00972-e73d-4884-adb0-a4280e79f7db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.343801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1120/2242] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:16.348216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aa01d-920c-4e3b-baaf-edb06ceaddbd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.357919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3aa01d-920c-4e3b-baaf-edb06ceaddbd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.362638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9b600-a07e-4c59-b136-fe7eb751cf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.363290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9b600-a07e-4c59-b136-fe7eb751cf97 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.363596 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9b600-a07e-4c59-b136-fe7eb751cf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.364386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b9b600-a07e-4c59-b136-fe7eb751cf97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.364589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9b600-a07e-4c59-b136-fe7eb751cf97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:26:16.364890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9b600-a07e-4c59-b136-fe7eb751cf97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.366504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b9b600-a07e-4c59-b136-fe7eb751cf97 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.379911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1123/2243] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:26:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:26:16.383617 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f00972-e73d-4884-adb0-a4280e79f7db None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.393728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f00972-e73d-4884-adb0-a4280e79f7db None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.398361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926eda6-fe58-4b05-b43f-224d0d4d6dff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.399044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926eda6-fe58-4b05-b43f-224d0d4d6dff admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.399331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926eda6-fe58-4b05-b43f-224d0d4d6dff admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.399609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b926eda6-fe58-4b05-b43f-224d0d4d6dff admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.400303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926eda6-fe58-4b05-b43f-224d0d4d6dff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.402365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b926eda6-fe58-4b05-b43f-224d0d4d6dff admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.427893 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1121/2244] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/projects => generated 385 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:26:16.432581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9b600-a07e-4c59-b136-fe7eb751cf97 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.442661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b9b600-a07e-4c59-b136-fe7eb751cf97 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.495290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:DELETE=3,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:16.504413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fd97f-0c19-4f45-87e2-0871add1296f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.505273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd97f-0c19-4f45-87e2-0871add1296f admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.505787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd97f-0c19-4f45-87e2-0871add1296f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.506131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22fd97f-0c19-4f45-87e2-0871add1296f admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.507064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fd97f-0c19-4f45-87e2-0871add1296f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.509044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22fd97f-0c19-4f45-87e2-0871add1296f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.529349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1124/2245] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:16.533411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926eda6-fe58-4b05-b43f-224d0d4d6dff admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.543737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b926eda6-fe58-4b05-b43f-224d0d4d6dff admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.548326 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2be48d-367c-4768-ac93-3296be02da8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.548979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2be48d-367c-4768-ac93-3296be02da8f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.549260 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2be48d-367c-4768-ac93-3296be02da8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.549555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2be48d-367c-4768-ac93-3296be02da8f admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.550279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2be48d-367c-4768-ac93-3296be02da8f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.551861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2be48d-367c-4768-ac93-3296be02da8f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.563045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1122/2246] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:16.566981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fd97f-0c19-4f45-87e2-0871add1296f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.578182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22fd97f-0c19-4f45-87e2-0871add1296f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.583164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e03095-58cc-4393-bfa9-6d16fc0193d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.583536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e03095-58cc-4393-bfa9-6d16fc0193d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.583823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e03095-58cc-4393-bfa9-6d16fc0193d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.584102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e03095-58cc-4393-bfa9-6d16fc0193d8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.584985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e03095-58cc-4393-bfa9-6d16fc0193d8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.586482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e03095-58cc-4393-bfa9-6d16fc0193d8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.600810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1125/2247] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:16.602023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2be48d-367c-4768-ac93-3296be02da8f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.612820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2be48d-367c-4768-ac93-3296be02da8f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.617607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdea208-d035-4e91-af99-592c9f738e9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.618478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdea208-d035-4e91-af99-592c9f738e9f admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.618757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdea208-d035-4e91-af99-592c9f738e9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.619028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdea208-d035-4e91-af99-592c9f738e9f admin admin] PATH_INFO: `/v3/projects/fe3ae99e3cb741539249a6bc56185ac7/users/283846b0f0e0404ea9bfae6dad50b13b/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.635254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdea208-d035-4e91-af99-592c9f738e9f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe3ae99e3cb741539249a6bc56185ac7, user_id=283846b0f0e0404ea9bfae6dad50b13b, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.638689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdea208-d035-4e91-af99-592c9f738e9f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.652348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1123/2248] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:16 2022] PUT /identity/v3/projects/fe3ae99e3cb741539249a6bc56185ac7/users/283846b0f0e0404ea9bfae6dad50b13b/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:16.662725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b34937-b276-4821-b65d-ec3b359b4b29 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.663018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b34937-b276-4821-b65d-ec3b359b4b29 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.663301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b34937-b276-4821-b65d-ec3b359b4b29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.690626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-31b34937-b276-4821-b65d-ec3b359b4b29 None None] MFA Rules not processed for user `283846b0f0e0404ea9bfae6dad50b13b`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:16.691852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-31b34937-b276-4821-b65d-ec3b359b4b29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.763510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1126/2249] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:16.775004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdea208-d035-4e91-af99-592c9f738e9f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:16.776848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9cdea208-d035-4e91-af99-592c9f738e9f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:16.839676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdea208-d035-4e91-af99-592c9f738e9f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:16.843737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ded18f-c5ae-4354-9e72-33ef45112eb7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:16.844470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ded18f-c5ae-4354-9e72-33ef45112eb7 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:16.844749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ded18f-c5ae-4354-9e72-33ef45112eb7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:16.844947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ded18f-c5ae-4354-9e72-33ef45112eb7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:16.854724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ded18f-c5ae-4354-9e72-33ef45112eb7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:16.856450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ded18f-c5ae-4354-9e72-33ef45112eb7 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:16.865887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1124/2250] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:24.686827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b34937-b276-4821-b65d-ec3b359b4b29 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:24.752825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b34937-b276-4821-b65d-ec3b359b4b29 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:24.757414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:24.758322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:24.758600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:24.758953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] PATH_INFO: `/v3/users/283846b0f0e0404ea9bfae6dad50b13b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:24.760853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=283846b0f0e0404ea9bfae6dad50b13b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:24.763012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:24.810678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '283846b0f0e0404ea9bfae6dad50b13b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:24.822298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '283846b0f0e0404ea9bfae6dad50b13b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:24.823679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:24.830491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '283846b0f0e0404ea9bfae6dad50b13b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:24.831414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '283846b0f0e0404ea9bfae6dad50b13b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:24.832364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '283846b0f0e0404ea9bfae6dad50b13b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:24.845832 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1127/2251] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:24 2022] DELETE /identity/v3/users/283846b0f0e0404ea9bfae6dad50b13b => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:24.850450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ef119-58da-4193-b222-c014c7d15138 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:24.850957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ef119-58da-4193-b222-c014c7d15138 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:24.850957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482ef119-58da-4193-b222-c014c7d15138 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:24.868315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-482ef119-58da-4193-b222-c014c7d15138 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:24.869559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-482ef119-58da-4193-b222-c014c7d15138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:24.902920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:24.929398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1125/2252] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:26:24.939526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:24.940942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:24.999779 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32f60b-e668-4811-88da-3427fc0a1130 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:25.003536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2669e-aafd-4e22-8be9-1cd7d6807c73 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:25.004184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2669e-aafd-4e22-8be9-1cd7d6807c73 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:25.004474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2669e-aafd-4e22-8be9-1cd7d6807c73 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:25.004754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c2669e-aafd-4e22-8be9-1cd7d6807c73 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:25.015848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2669e-aafd-4e22-8be9-1cd7d6807c73 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:25.017671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45c2669e-aafd-4e22-8be9-1cd7d6807c73 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:25.028547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1128/2253] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:25.445606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ef119-58da-4193-b222-c014c7d15138 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:25.447083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-482ef119-58da-4193-b222-c014c7d15138 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:25.459731 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482ef119-58da-4193-b222-c014c7d15138 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:25.464416 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:25.465291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:25.465878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:25.466156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] PATH_INFO: `/v3/users/ea26a71427d14ab08a84f2710d9ad0bc` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:25.467350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ea26a71427d14ab08a84f2710d9ad0bc)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:25.469288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:25.524088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ea26a71427d14ab08a84f2710d9ad0bc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:25.539884 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ea26a71427d14ab08a84f2710d9ad0bc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:25.549358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ea26a71427d14ab08a84f2710d9ad0bc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:25.550200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ea26a71427d14ab08a84f2710d9ad0bc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:25.550749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-8b9f3cb8-830f-4682-bb05-33ce111646c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ea26a71427d14ab08a84f2710d9ad0bc'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:25.565803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2669e-aafd-4e22-8be9-1cd7d6807c73 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:25.567245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-45c2669e-aafd-4e22-8be9-1cd7d6807c73 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:25.571135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1126/2254] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:25 2022] DELETE /identity/v3/users/ea26a71427d14ab08a84f2710d9ad0bc => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:25.576169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982137b-7cdb-449c-ab20-9b20ad78b8ad None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:25.576465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982137b-7cdb-449c-ab20-9b20ad78b8ad None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:25.576748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4982137b-7cdb-449c-ab20-9b20ad78b8ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:25.597580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c2669e-aafd-4e22-8be9-1cd7d6807c73 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:25.602065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4982137b-7cdb-449c-ab20-9b20ad78b8ad None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:25.606397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4982137b-7cdb-449c-ab20-9b20ad78b8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:25.663305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:25.663723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:25.664107 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:25.664454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] PATH_INFO: `/v3/projects/fe3ae99e3cb741539249a6bc56185ac7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:25.665759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe3ae99e3cb741539249a6bc56185ac7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:25.668419 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:25.711457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1127/2255] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:26:25.722089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982137b-7cdb-449c-ab20-9b20ad78b8ad None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:25.723255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4982137b-7cdb-449c-ab20-9b20ad78b8ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:25.761558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe3ae99e3cb741539249a6bc56185ac7', 'request_id': 'req-b463f57b-c502-41e5-a68d-9660298c8310'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:25.773197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe3ae99e3cb741539249a6bc56185ac7', 'request_id': 'req-b463f57b-c502-41e5-a68d-9660298c8310'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:25.780337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe3ae99e3cb741539249a6bc56185ac7', 'request_id': 'req-b463f57b-c502-41e5-a68d-9660298c8310'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:25.808982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] The token cache is being invalidate because project fe3ae99e3cb741539249a6bc56185ac7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:25.809287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:25.811193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b463f57b-c502-41e5-a68d-9660298c8310 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:25.819976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1129/2256] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:25 2022] DELETE /identity/v3/projects/fe3ae99e3cb741539249a6bc56185ac7 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:25.820152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704abf52-eeba-4402-a54a-cb8d731cd842 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:25.820401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704abf52-eeba-4402-a54a-cb8d731cd842 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:25.820682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704abf52-eeba-4402-a54a-cb8d731cd842 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:25.855867 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-704abf52-eeba-4402-a54a-cb8d731cd842 None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:25.857410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-704abf52-eeba-4402-a54a-cb8d731cd842 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:25.871281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:25.872387 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4982137b-7cdb-449c-ab20-9b20ad78b8ad None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:25.880843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:25.952049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1130/2257] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 19:26:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:25.954415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6a526-076e-4002-8f11-d6c7e694d15c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:25.954793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6a526-076e-4002-8f11-d6c7e694d15c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:25.955204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a6a526-076e-4002-8f11-d6c7e694d15c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:25.987941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a1a6a526-076e-4002-8f11-d6c7e694d15c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:25.989477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a1a6a526-076e-4002-8f11-d6c7e694d15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:26.000422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172437e5-4060-49cc-a2eb-47d7ddfbb10f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:26.001227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172437e5-4060-49cc-a2eb-47d7ddfbb10f service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.001545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172437e5-4060-49cc-a2eb-47d7ddfbb10f service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.001830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172437e5-4060-49cc-a2eb-47d7ddfbb10f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.007472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-172437e5-4060-49cc-a2eb-47d7ddfbb10f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:26.065088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1131/2258] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:26.066309 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6a526-076e-4002-8f11-d6c7e694d15c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:26.067613 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a1a6a526-076e-4002-8f11-d6c7e694d15c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:26.091379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172437e5-4060-49cc-a2eb-47d7ddfbb10f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:26.092818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172437e5-4060-49cc-a2eb-47d7ddfbb10f service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:26.102511 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1128/2259] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:26.104355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a9ce-5a40-4fab-8ca2-68de35a5591e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.104826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a9ce-5a40-4fab-8ca2-68de35a5591e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.105280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9a5a9ce-5a40-4fab-8ca2-68de35a5591e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.123643 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f9a5a9ce-5a40-4fab-8ca2-68de35a5591e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:26.124851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f9a5a9ce-5a40-4fab-8ca2-68de35a5591e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:26.133557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1129/2260] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:26.137936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5a9ce-5a40-4fab-8ca2-68de35a5591e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:26.147421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9a5a9ce-5a40-4fab-8ca2-68de35a5591e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:26.148271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a6a526-076e-4002-8f11-d6c7e694d15c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:26.153427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e5ec8-afcd-4762-b5a6-0329d7085150 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:26.154197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e5ec8-afcd-4762-b5a6-0329d7085150 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.154555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e5ec8-afcd-4762-b5a6-0329d7085150 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.154787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342e5ec8-afcd-4762-b5a6-0329d7085150 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.155321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fac55-4b60-466c-b1a4-23bd0abaa0d6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:26.155622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e5ec8-afcd-4762-b5a6-0329d7085150 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:26:26.155931 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e5ec8-afcd-4762-b5a6-0329d7085150 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:26.156059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fac55-4b60-466c-b1a4-23bd0abaa0d6 service swift] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.156248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fac55-4b60-466c-b1a4-23bd0abaa0d6 service swift] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.156499 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-390fac55-4b60-466c-b1a4-23bd0abaa0d6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.158331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342e5ec8-afcd-4762-b5a6-0329d7085150 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:26.166194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fac55-4b60-466c-b1a4-23bd0abaa0d6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:26.168565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-390fac55-4b60-466c-b1a4-23bd0abaa0d6 service swift] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:26.179235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1130/2261] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:26:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 19:26:26.180096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1132/2262] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 19:26:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:26:26.186211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e5ec8-afcd-4762-b5a6-0329d7085150 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:26.196474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342e5ec8-afcd-4762-b5a6-0329d7085150 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:26.202554 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cfe95-b9c7-4c46-b564-6a12c55df333 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:26.203488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cfe95-b9c7-4c46-b564-6a12c55df333 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.203787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cfe95-b9c7-4c46-b564-6a12c55df333 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.204263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6cfe95-b9c7-4c46-b564-6a12c55df333 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.205063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cfe95-b9c7-4c46-b564-6a12c55df333 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:26.207519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6cfe95-b9c7-4c46-b564-6a12c55df333 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:26.242750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1131/2263] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:26 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:26.243964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fac55-4b60-466c-b1a4-23bd0abaa0d6 service swift] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:26.255908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-390fac55-4b60-466c-b1a4-23bd0abaa0d6 service swift] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:26.348253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:26.369253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831a505-a985-451e-a5d7-ff49cecafcbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:26.371004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831a505-a985-451e-a5d7-ff49cecafcbc admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.371521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831a505-a985-451e-a5d7-ff49cecafcbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.371951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c831a505-a985-451e-a5d7-ff49cecafcbc admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.372660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831a505-a985-451e-a5d7-ff49cecafcbc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:26.375062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c831a505-a985-451e-a5d7-ff49cecafcbc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:26.400403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1133/2264] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:26 2022] POST /identity/v3/users => generated 282 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:26:26.402569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cfe95-b9c7-4c46-b564-6a12c55df333 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:26.420873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6cfe95-b9c7-4c46-b564-6a12c55df333 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:26.429202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef04a2b-9a18-41ca-ad89-b5d46ed8175c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:26.430389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef04a2b-9a18-41ca-ad89-b5d46ed8175c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.431118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef04a2b-9a18-41ca-ad89-b5d46ed8175c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.431415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef04a2b-9a18-41ca-ad89-b5d46ed8175c admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.432845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef04a2b-9a18-41ca-ad89-b5d46ed8175c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:26.437817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef04a2b-9a18-41ca-ad89-b5d46ed8175c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:26.457557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1132/2265] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:26.464896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831a505-a985-451e-a5d7-ff49cecafcbc admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:26.471745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef04a2b-9a18-41ca-ad89-b5d46ed8175c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:26.475842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0ef04a2b-9a18-41ca-ad89-b5d46ed8175c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:26.494204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c831a505-a985-451e-a5d7-ff49cecafcbc admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:26.502950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef04a2b-9a18-41ca-ad89-b5d46ed8175c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:26.505057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe61c00-ca5e-40d3-a388-6e34cef6b0df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:26.507338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe61c00-ca5e-40d3-a388-6e34cef6b0df admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.507629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe61c00-ca5e-40d3-a388-6e34cef6b0df admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.508250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:26.509259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.509387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe61c00-ca5e-40d3-a388-6e34cef6b0df admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.509645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.509938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] PATH_INFO: `/v3/projects/ed168e10a8f54c5dab88fb1b5fc48c8c` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.510683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe61c00-ca5e-40d3-a388-6e34cef6b0df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:26.512237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe61c00-ca5e-40d3-a388-6e34cef6b0df admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:26.512468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed168e10a8f54c5dab88fb1b5fc48c8c)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:26.515064 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:26.529552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1134/2266] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:26 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:26:26.530863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe61c00-ca5e-40d3-a388-6e34cef6b0df admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:26.542016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe61c00-ca5e-40d3-a388-6e34cef6b0df admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:26.547269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50aab453-1d3d-4ae5-8e5f-b7043dca5025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:26.548005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aab453-1d3d-4ae5-8e5f-b7043dca5025 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.548455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aab453-1d3d-4ae5-8e5f-b7043dca5025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.548742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50aab453-1d3d-4ae5-8e5f-b7043dca5025 admin admin] PATH_INFO: `/v3/projects/7be36032c7b5403682c9eb90a34d23a2/users/b0594cb7f3dc4574b022efcbbb97c288/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.566950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aab453-1d3d-4ae5-8e5f-b7043dca5025 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7be36032c7b5403682c9eb90a34d23a2, user_id=b0594cb7f3dc4574b022efcbbb97c288, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:26.569997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50aab453-1d3d-4ae5-8e5f-b7043dca5025 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:26.578450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed168e10a8f54c5dab88fb1b5fc48c8c', 'request_id': 'req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:26.585121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed168e10a8f54c5dab88fb1b5fc48c8c', 'request_id': 'req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:26.586953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1135/2267] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:26 2022] PUT /identity/v3/projects/7be36032c7b5403682c9eb90a34d23a2/users/b0594cb7f3dc4574b022efcbbb97c288/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:26:26.587822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:INSERT=2,keystone:DELETE=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:26.600758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85f12-97da-4dc9-a943-2bf8c88b4894 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.601073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85f12-97da-4dc9-a943-2bf8c88b4894 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.601311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c85f12-97da-4dc9-a943-2bf8c88b4894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.601791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed168e10a8f54c5dab88fb1b5fc48c8c', 'request_id': 'req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:26.614329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] The token cache is being invalidate because project ed168e10a8f54c5dab88fb1b5fc48c8c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:26.614988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:26.615773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:26.617559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1133/2268] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:26 2022] DELETE /identity/v3/projects/ed168e10a8f54c5dab88fb1b5fc48c8c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:26.633691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-34c85f12-97da-4dc9-a943-2bf8c88b4894 None None] MFA Rules not processed for user `b0594cb7f3dc4574b022efcbbb97c288`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:26.636070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-34c85f12-97da-4dc9-a943-2bf8c88b4894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:26.725129 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1136/2269] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:26 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:26.742743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:26.744173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:26.819282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbc0f7f-7eb3-488b-8b01-86f0e0b9e279 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:26.872941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ba374-0bd1-4dd5-b913-5f0df3b6c996 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:26.873648 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ba374-0bd1-4dd5-b913-5f0df3b6c996 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.873951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ba374-0bd1-4dd5-b913-5f0df3b6c996 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.874236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-333ba374-0bd1-4dd5-b913-5f0df3b6c996 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.883445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ba374-0bd1-4dd5-b913-5f0df3b6c996 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:26.884872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-333ba374-0bd1-4dd5-b913-5f0df3b6c996 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:26.894101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1134/2270] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:26 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:26.947253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab04b-c08d-475e-8d28-bec271355b63 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:26.947723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab04b-c08d-475e-8d28-bec271355b63 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:26.947968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20ab04b-c08d-475e-8d28-bec271355b63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:26.966781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a20ab04b-c08d-475e-8d28-bec271355b63 None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:26.967986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a20ab04b-c08d-475e-8d28-bec271355b63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:27.028260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1137/2271] 10.222.0.69 () {58 vars in 967 bytes} [Wed Jul 20 19:26:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:27.042788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ba374-0bd1-4dd5-b913-5f0df3b6c996 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:27.044805 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-333ba374-0bd1-4dd5-b913-5f0df3b6c996 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:27.130941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-333ba374-0bd1-4dd5-b913-5f0df3b6c996 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:27.137799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61d967-d414-4d7f-b8f1-a7136c063698 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:27.138466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61d967-d414-4d7f-b8f1-a7136c063698 service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:27.138759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61d967-d414-4d7f-b8f1-a7136c063698 service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:27.139174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c61d967-d414-4d7f-b8f1-a7136c063698 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:27.155480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61d967-d414-4d7f-b8f1-a7136c063698 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:27.157346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c61d967-d414-4d7f-b8f1-a7136c063698 service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:27.179914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1135/2272] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 19:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:26:27.447472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86281d6a-9cba-4927-b71b-6872a9cf3ecf None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:27.448029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86281d6a-9cba-4927-b71b-6872a9cf3ecf None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:27.448273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86281d6a-9cba-4927-b71b-6872a9cf3ecf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:27.471947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-86281d6a-9cba-4927-b71b-6872a9cf3ecf None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:27.473134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-86281d6a-9cba-4927-b71b-6872a9cf3ecf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:27.481245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1138/2273] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:27.499232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61d967-d414-4d7f-b8f1-a7136c063698 service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:27.510926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c61d967-d414-4d7f-b8f1-a7136c063698 service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:27.516144 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed1067-7052-40bd-94f4-13bc22b7bb11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:27.516950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed1067-7052-40bd-94f4-13bc22b7bb11 service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:27.517304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed1067-7052-40bd-94f4-13bc22b7bb11 service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:27.517684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78ed1067-7052-40bd-94f4-13bc22b7bb11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:27.527018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed1067-7052-40bd-94f4-13bc22b7bb11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:27.528666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78ed1067-7052-40bd-94f4-13bc22b7bb11 service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:27.537789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1136/2274] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 19:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:26:27.882477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078d3bc-3257-4937-a866-1c72f95a9b0d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:27.882769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078d3bc-3257-4937-a866-1c72f95a9b0d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:27.883053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b078d3bc-3257-4937-a866-1c72f95a9b0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:27.902709 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b078d3bc-3257-4937-a866-1c72f95a9b0d None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:27.903918 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b078d3bc-3257-4937-a866-1c72f95a9b0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:27.911785 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1139/2275] 10.222.0.69 () {58 vars in 967 bytes} [Wed Jul 20 19:26:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:27.924741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed1067-7052-40bd-94f4-13bc22b7bb11 service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:27.936253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78ed1067-7052-40bd-94f4-13bc22b7bb11 service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:27.941307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3166c5-dd0c-4ced-b313-70653e7bf057 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:27.941987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3166c5-dd0c-4ced-b313-70653e7bf057 service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:27.942345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3166c5-dd0c-4ced-b313-70653e7bf057 service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:27.942641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b3166c5-dd0c-4ced-b313-70653e7bf057 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:27.951360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3166c5-dd0c-4ced-b313-70653e7bf057 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:27.952930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b3166c5-dd0c-4ced-b313-70653e7bf057 service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:27.961128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1137/2276] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 19:26:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 19:26:34.771161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9ff11-7034-412d-b0c1-22db34ecce55 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:34.771628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9ff11-7034-412d-b0c1-22db34ecce55 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:34.771883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf9ff11-7034-412d-b0c1-22db34ecce55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:34.791694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5cf9ff11-7034-412d-b0c1-22db34ecce55 None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:34.792912 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5cf9ff11-7034-412d-b0c1-22db34ecce55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:34.801304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1140/2277] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 19:26:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:26:34.813895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3166c5-dd0c-4ced-b313-70653e7bf057 service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:34.825530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b3166c5-dd0c-4ced-b313-70653e7bf057 service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:34.830596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08e26459-b8d5-42a4-8668-f990fab35c16 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:34.831291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e26459-b8d5-42a4-8668-f990fab35c16 service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:34.831749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e26459-b8d5-42a4-8668-f990fab35c16 service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:34.832030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08e26459-b8d5-42a4-8668-f990fab35c16 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:34.841878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e26459-b8d5-42a4-8668-f990fab35c16 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:34.843524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08e26459-b8d5-42a4-8668-f990fab35c16 service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:34.847599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=1,keystone:SELECT=99 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:34.854899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1138/2278] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 19:26:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:26:36.494209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9ff11-7034-412d-b0c1-22db34ecce55 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:36.503311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5cf9ff11-7034-412d-b0c1-22db34ecce55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:36.572441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:36.645635 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf9ff11-7034-412d-b0c1-22db34ecce55 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:36.651985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:36.653346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:36.654619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:36.654619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] PATH_INFO: `/v3/users/67416d4eea6946fdb77a09242e58a12f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:36.656422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=67416d4eea6946fdb77a09242e58a12f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:36.660444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:36.757402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67416d4eea6946fdb77a09242e58a12f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:36.775713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67416d4eea6946fdb77a09242e58a12f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:36.784921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67416d4eea6946fdb77a09242e58a12f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:36.785979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67416d4eea6946fdb77a09242e58a12f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:36.786905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67416d4eea6946fdb77a09242e58a12f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:36.803065 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1141/2279] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:36 2022] DELETE /identity/v3/users/67416d4eea6946fdb77a09242e58a12f => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:36.808009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debebd8-9ea5-4359-90dc-d2d7b00a83a4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:36.809037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debebd8-9ea5-4359-90dc-d2d7b00a83a4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:36.810131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0debebd8-9ea5-4359-90dc-d2d7b00a83a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:36.831066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0debebd8-9ea5-4359-90dc-d2d7b00a83a4 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:36.832601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0debebd8-9ea5-4359-90dc-d2d7b00a83a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:36.913484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1139/2280] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:26:36.924447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:36.925914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:36.997285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd71ed3-709a-4a3e-9486-0574294dba23 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:37.001089 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21216652-9b5e-4886-b157-1f00fa286a37 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:37.003971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21216652-9b5e-4886-b157-1f00fa286a37 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:37.004254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21216652-9b5e-4886-b157-1f00fa286a37 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:37.004530 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21216652-9b5e-4886-b157-1f00fa286a37 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:37.016173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21216652-9b5e-4886-b157-1f00fa286a37 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:37.017609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21216652-9b5e-4886-b157-1f00fa286a37 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:37.030022 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1142/2281] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:37.245920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debebd8-9ea5-4359-90dc-d2d7b00a83a4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:37.247704 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0debebd8-9ea5-4359-90dc-d2d7b00a83a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:37.276857 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0debebd8-9ea5-4359-90dc-d2d7b00a83a4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:37.288528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:37.290045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:37.290339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:37.291379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] PATH_INFO: `/v3/projects/d66a7bac21e44b72a59bb74c917daae1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:37.294013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] RBAC: Authorizing `identity:delete_project(project_id=d66a7bac21e44b72a59bb74c917daae1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:37.296120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:37.377973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd66a7bac21e44b72a59bb74c917daae1', 'request_id': 'req-871b1c7f-2f4a-42e5-abc4-577465c09a9f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:37.386331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd66a7bac21e44b72a59bb74c917daae1', 'request_id': 'req-871b1c7f-2f4a-42e5-abc4-577465c09a9f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:37.447523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd66a7bac21e44b72a59bb74c917daae1', 'request_id': 'req-871b1c7f-2f4a-42e5-abc4-577465c09a9f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:37.472532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] The token cache is being invalidate because project d66a7bac21e44b72a59bb74c917daae1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:37.472849 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:37.473870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-871b1c7f-2f4a-42e5-abc4-577465c09a9f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:37.475735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1140/2282] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:37 2022] DELETE /identity/v3/projects/d66a7bac21e44b72a59bb74c917daae1 => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:37.481890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5585-a6de-4771-854c-8b5da889f66d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:37.482641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5585-a6de-4771-854c-8b5da889f66d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:37.483381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bc5585-a6de-4771-854c-8b5da889f66d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:37.510787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-50bc5585-a6de-4771-854c-8b5da889f66d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:37.513180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-50bc5585-a6de-4771-854c-8b5da889f66d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:37.575487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:37.605733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1143/2283] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:37.613035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dc7f9-1855-47c2-b145-b1baeee13f2c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:37.613385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dc7f9-1855-47c2-b145-b1baeee13f2c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:37.613639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750dc7f9-1855-47c2-b145-b1baeee13f2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:37.640861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-750dc7f9-1855-47c2-b145-b1baeee13f2c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:37.642143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-750dc7f9-1855-47c2-b145-b1baeee13f2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:37.658246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1141/2284] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:37.659917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5585-a6de-4771-854c-8b5da889f66d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:37.674300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bc5585-a6de-4771-854c-8b5da889f66d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:37.682246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fca79-969c-4296-a3ad-82fca055d50e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:37.683346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fca79-969c-4296-a3ad-82fca055d50e admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:37.683893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fca79-969c-4296-a3ad-82fca055d50e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:37.685025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a02fca79-969c-4296-a3ad-82fca055d50e admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:37.686084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02fca79-969c-4296-a3ad-82fca055d50e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:26:37.686620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02fca79-969c-4296-a3ad-82fca055d50e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:37.688576 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a02fca79-969c-4296-a3ad-82fca055d50e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:37.707549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dc7f9-1855-47c2-b145-b1baeee13f2c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:37.713898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1144/2285] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:26:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:26:37.719017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750dc7f9-1855-47c2-b145-b1baeee13f2c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:37.724788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1f5ec-64fb-4b9e-98dc-c093a6f7d02e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:37.725838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1f5ec-64fb-4b9e-98dc-c093a6f7d02e admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:37.726343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1f5ec-64fb-4b9e-98dc-c093a6f7d02e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:37.726916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e1f5ec-64fb-4b9e-98dc-c093a6f7d02e admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:37.728459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1f5ec-64fb-4b9e-98dc-c093a6f7d02e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:37.731713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e1f5ec-64fb-4b9e-98dc-c093a6f7d02e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:37.765274 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1142/2286] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:37 2022] POST /identity/v3/projects => generated 383 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:26:37.769173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fca79-969c-4296-a3ad-82fca055d50e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:37.780909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a02fca79-969c-4296-a3ad-82fca055d50e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:37.877808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efd227-6a55-4246-89c6-3716056d36f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:37.878821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efd227-6a55-4246-89c6-3716056d36f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:37.878944 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efd227-6a55-4246-89c6-3716056d36f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:37.879215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9efd227-6a55-4246-89c6-3716056d36f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:37.880140 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efd227-6a55-4246-89c6-3716056d36f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:37.882874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9efd227-6a55-4246-89c6-3716056d36f7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:37.917324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1f5ec-64fb-4b9e-98dc-c093a6f7d02e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:37.920645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1145/2287] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:37 2022] POST /identity/v3/users => generated 292 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:37.946695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e1f5ec-64fb-4b9e-98dc-c093a6f7d02e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:37.960816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55727074-00f6-46e7-b2d0-5efb78b48405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:37.960816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727074-00f6-46e7-b2d0-5efb78b48405 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:37.960816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727074-00f6-46e7-b2d0-5efb78b48405 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:37.960816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55727074-00f6-46e7-b2d0-5efb78b48405 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:37.961690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55727074-00f6-46e7-b2d0-5efb78b48405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:37.967791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55727074-00f6-46e7-b2d0-5efb78b48405 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:37.981573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1143/2288] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 63 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:37.984813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efd227-6a55-4246-89c6-3716056d36f7 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:37.998708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9efd227-6a55-4246-89c6-3716056d36f7 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:38.003926 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-001cc993-1f0a-4aba-b809-ab319f7225a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:38.004713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cc993-1f0a-4aba-b809-ab319f7225a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:38.005071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cc993-1f0a-4aba-b809-ab319f7225a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:38.005444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-001cc993-1f0a-4aba-b809-ab319f7225a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:38.006297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cc993-1f0a-4aba-b809-ab319f7225a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:38.008525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-001cc993-1f0a-4aba-b809-ab319f7225a4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:38.021173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1146/2289] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:37 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:26:38.027713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55727074-00f6-46e7-b2d0-5efb78b48405 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:38.043654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55727074-00f6-46e7-b2d0-5efb78b48405 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:38.056354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac86e60-fddd-41cd-9f9c-9bf87115341b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:38.057817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac86e60-fddd-41cd-9f9c-9bf87115341b admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:38.058103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac86e60-fddd-41cd-9f9c-9bf87115341b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:38.058435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac86e60-fddd-41cd-9f9c-9bf87115341b admin admin] PATH_INFO: `/v3/projects/e0bb4af78c1d4589b1d3cc90d5e6be63/users/78ae826a812b44328c48b82e9498ad8b/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:38.076301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac86e60-fddd-41cd-9f9c-9bf87115341b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e0bb4af78c1d4589b1d3cc90d5e6be63, user_id=78ae826a812b44328c48b82e9498ad8b, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:38.079043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac86e60-fddd-41cd-9f9c-9bf87115341b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:38.092519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1144/2290] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:38 2022] PUT /identity/v3/projects/e0bb4af78c1d4589b1d3cc90d5e6be63/users/78ae826a812b44328c48b82e9498ad8b/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:38.097933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2d1a5-a310-4da8-8e02-1d046aa4e081 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:38.098235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2d1a5-a310-4da8-8e02-1d046aa4e081 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:38.098518 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad2d1a5-a310-4da8-8e02-1d046aa4e081 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:38.120048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3ad2d1a5-a310-4da8-8e02-1d046aa4e081 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:38.121316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3ad2d1a5-a310-4da8-8e02-1d046aa4e081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:38.213003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1147/2291] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:38.220116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac86e60-fddd-41cd-9f9c-9bf87115341b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:38.222411 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fac86e60-fddd-41cd-9f9c-9bf87115341b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:38.247962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:38.296711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac86e60-fddd-41cd-9f9c-9bf87115341b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:38.302823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd5b1fc8-cb1f-4aec-9afc-ed7accfdbf8b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:38.303768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b1fc8-cb1f-4aec-9afc-ed7accfdbf8b service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:38.304307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b1fc8-cb1f-4aec-9afc-ed7accfdbf8b service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:38.304823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd5b1fc8-cb1f-4aec-9afc-ed7accfdbf8b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:38.342762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2d1a5-a310-4da8-8e02-1d046aa4e081 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:38.343351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b1fc8-cb1f-4aec-9afc-ed7accfdbf8b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:38.344303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3ad2d1a5-a310-4da8-8e02-1d046aa4e081 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:38.346868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd5b1fc8-cb1f-4aec-9afc-ed7accfdbf8b service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:38.375837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1145/2292] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:38.446737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:38.491246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad2d1a5-a310-4da8-8e02-1d046aa4e081 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:38.499490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd6710-96ad-481c-985c-7b24b060d70e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:38.500185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd6710-96ad-481c-985c-7b24b060d70e service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:38.500674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd6710-96ad-481c-985c-7b24b060d70e service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:38.500842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fd6710-96ad-481c-985c-7b24b060d70e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:38.502763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e3fd6710-96ad-481c-985c-7b24b060d70e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:38.518121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd6710-96ad-481c-985c-7b24b060d70e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:38.521285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3fd6710-96ad-481c-985c-7b24b060d70e service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:38.538299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1148/2293] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:26:39.446309 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd31205-8dae-44d2-9337-9d4399f17983 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:39.447363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd31205-8dae-44d2-9337-9d4399f17983 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:39.447817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebd31205-8dae-44d2-9337-9d4399f17983 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:39.507635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-ebd31205-8dae-44d2-9337-9d4399f17983 None None] MFA Rules not processed for user `78ae826a812b44328c48b82e9498ad8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:39.510276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ebd31205-8dae-44d2-9337-9d4399f17983 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:39.638992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1146/2294] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:26:39.651037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd6710-96ad-481c-985c-7b24b060d70e service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:39.653711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e3fd6710-96ad-481c-985c-7b24b060d70e service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:39.791637 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fd6710-96ad-481c-985c-7b24b060d70e service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:39.796853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427adae0-c9d1-44ea-be55-c622c9655258 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:39.797880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427adae0-c9d1-44ea-be55-c622c9655258 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:39.798355 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427adae0-c9d1-44ea-be55-c622c9655258 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:39.798830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427adae0-c9d1-44ea-be55-c622c9655258 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:39.811188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427adae0-c9d1-44ea-be55-c622c9655258 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:39.813170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-427adae0-c9d1-44ea-be55-c622c9655258 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:39.824699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1149/2295] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:26:42.500879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d288f-d969-4bcf-99ac-ce3eaea31235 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:42.501464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d288f-d969-4bcf-99ac-ce3eaea31235 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:42.501820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939d288f-d969-4bcf-99ac-ce3eaea31235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:42.520680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-939d288f-d969-4bcf-99ac-ce3eaea31235 None None] MFA Rules not processed for user `78ae826a812b44328c48b82e9498ad8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:42.522071 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-939d288f-d969-4bcf-99ac-ce3eaea31235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:42.530470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1147/2296] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:42 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:49.822956 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=36 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:52.514861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=53 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:53.049061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40799c84-9e3b-4768-8588-41c8305366c5 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:53.050336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40799c84-9e3b-4768-8588-41c8305366c5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:53.050898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40799c84-9e3b-4768-8588-41c8305366c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:53.073796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-40799c84-9e3b-4768-8588-41c8305366c5 None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:53.075418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-40799c84-9e3b-4768-8588-41c8305366c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:53.144818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1150/2297] 10.222.0.69 () {58 vars in 967 bytes} [Wed Jul 20 19:26:53 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:53.153185 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d288f-d969-4bcf-99ac-ce3eaea31235 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:53.155220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-939d288f-d969-4bcf-99ac-ce3eaea31235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:53.227511 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939d288f-d969-4bcf-99ac-ce3eaea31235 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:53.232702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab6ce9-874a-4043-87c9-4683e077245a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:53.233366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab6ce9-874a-4043-87c9-4683e077245a service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:53.233679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab6ce9-874a-4043-87c9-4683e077245a service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:53.233959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ab6ce9-874a-4043-87c9-4683e077245a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:53.241993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab6ce9-874a-4043-87c9-4683e077245a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:53.243597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ab6ce9-874a-4043-87c9-4683e077245a service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:53.252153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1148/2298] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 19:26:53 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:26:55.577777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40799c84-9e3b-4768-8588-41c8305366c5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:55.580190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-40799c84-9e3b-4768-8588-41c8305366c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:55.600235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40799c84-9e3b-4768-8588-41c8305366c5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:55.605925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:55.606903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:55.607438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:55.607723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] PATH_INFO: `/v3/users/afa79c7408524e979cf0e9a595fe731a` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:55.608968 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=afa79c7408524e979cf0e9a595fe731a)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:55.612063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:55.679270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'afa79c7408524e979cf0e9a595fe731a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:55.697573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'afa79c7408524e979cf0e9a595fe731a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:55.710390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'afa79c7408524e979cf0e9a595fe731a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:55.710808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'afa79c7408524e979cf0e9a595fe731a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:55.711827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-17040aae-ef4d-46f6-94b0-25acdb22c2bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'afa79c7408524e979cf0e9a595fe731a'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:55.732277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1151/2299] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:26:55 2022] DELETE /identity/v3/users/afa79c7408524e979cf0e9a595fe731a => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:55.932713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab6ce9-874a-4043-87c9-4683e077245a service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:55.946398 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ab6ce9-874a-4043-87c9-4683e077245a service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:56.038507 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:56.039430 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.039897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.040176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] PATH_INFO: `/v3/projects/a51cf9bd2ea44ebe8edd25559baafc4e` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.041407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a51cf9bd2ea44ebe8edd25559baafc4e)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:56.044155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:56.123989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a51cf9bd2ea44ebe8edd25559baafc4e', 'request_id': 'req-748670f7-b3a7-4ff9-aefe-fa861823281a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:56.133768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a51cf9bd2ea44ebe8edd25559baafc4e', 'request_id': 'req-748670f7-b3a7-4ff9-aefe-fa861823281a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:56.151532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a51cf9bd2ea44ebe8edd25559baafc4e', 'request_id': 'req-748670f7-b3a7-4ff9-aefe-fa861823281a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:56.168729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] The token cache is being invalidate because project a51cf9bd2ea44ebe8edd25559baafc4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:26:56.169023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:56.169717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-748670f7-b3a7-4ff9-aefe-fa861823281a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:26:56.171668 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1149/2300] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:26:55 2022] DELETE /identity/v3/projects/a51cf9bd2ea44ebe8edd25559baafc4e => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:56.177776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6868772-3dc9-4950-a981-36ae36644cae None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.178168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6868772-3dc9-4950-a981-36ae36644cae None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.178565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6868772-3dc9-4950-a981-36ae36644cae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.197970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-c6868772-3dc9-4950-a981-36ae36644cae None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:56.199362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c6868772-3dc9-4950-a981-36ae36644cae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:56.285559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1152/2301] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:56.287705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf240506-3f15-4938-88db-6cb0f71ebf9d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.288216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf240506-3f15-4938-88db-6cb0f71ebf9d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.288908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf240506-3f15-4938-88db-6cb0f71ebf9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.313151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cf240506-3f15-4938-88db-6cb0f71ebf9d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:56.319576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cf240506-3f15-4938-88db-6cb0f71ebf9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:56.329454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1150/2302] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:56.334570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6868772-3dc9-4950-a981-36ae36644cae None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:56.345949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6868772-3dc9-4950-a981-36ae36644cae None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:56.350626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84933539-20c0-4f6c-bd7e-02652a792264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:56.351290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933539-20c0-4f6c-bd7e-02652a792264 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.351579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933539-20c0-4f6c-bd7e-02652a792264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.351852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84933539-20c0-4f6c-bd7e-02652a792264 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.352619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84933539-20c0-4f6c-bd7e-02652a792264 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:26:56.352930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84933539-20c0-4f6c-bd7e-02652a792264 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:56.354644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84933539-20c0-4f6c-bd7e-02652a792264 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:56.371460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1153/2303] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:26:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:26:56.372899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf240506-3f15-4938-88db-6cb0f71ebf9d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:56.384118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf240506-3f15-4938-88db-6cb0f71ebf9d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:56.389239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff0e2e-cd54-4813-ab02-2beb250518c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:56.390226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff0e2e-cd54-4813-ab02-2beb250518c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.390519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff0e2e-cd54-4813-ab02-2beb250518c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.390801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aff0e2e-cd54-4813-ab02-2beb250518c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.391620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff0e2e-cd54-4813-ab02-2beb250518c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:56.393568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aff0e2e-cd54-4813-ab02-2beb250518c6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:56.405168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:56.427761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1151/2304] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:26:56 2022] POST /identity/v3/projects => generated 369 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:56.428840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84933539-20c0-4f6c-bd7e-02652a792264 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:56.435493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:26:56.439918 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84933539-20c0-4f6c-bd7e-02652a792264 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:56.504264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342762c-81f7-4438-b537-39cae26c7652 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:56.504942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342762c-81f7-4438-b537-39cae26c7652 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.505226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342762c-81f7-4438-b537-39cae26c7652 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.505504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342762c-81f7-4438-b537-39cae26c7652 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.506345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342762c-81f7-4438-b537-39cae26c7652 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:56.508332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342762c-81f7-4438-b537-39cae26c7652 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:56.530712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1154/2305] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:26:56 2022] POST /identity/v3/users => generated 285 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:26:56.534133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff0e2e-cd54-4813-ab02-2beb250518c6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:56.544953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aff0e2e-cd54-4813-ab02-2beb250518c6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:56.549729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ac9ce-78b9-4c26-a422-d088a5b43a3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:56.550395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ac9ce-78b9-4c26-a422-d088a5b43a3d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.550676 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ac9ce-78b9-4c26-a422-d088a5b43a3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.550951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9ac9ce-78b9-4c26-a422-d088a5b43a3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.551685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ac9ce-78b9-4c26-a422-d088a5b43a3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:56.553287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9ac9ce-78b9-4c26-a422-d088a5b43a3d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:56.565345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1152/2306] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 19:26:56.569598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342762c-81f7-4438-b537-39cae26c7652 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:56.582905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342762c-81f7-4438-b537-39cae26c7652 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:56.588677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce7cc6a-50db-48a5-8100-fb98669c80a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:56.590375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7cc6a-50db-48a5-8100-fb98669c80a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.592981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7cc6a-50db-48a5-8100-fb98669c80a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.592981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce7cc6a-50db-48a5-8100-fb98669c80a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.592981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7cc6a-50db-48a5-8100-fb98669c80a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:56.594531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce7cc6a-50db-48a5-8100-fb98669c80a8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:56.606244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1155/2307] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:26:56 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:26:56.610479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ac9ce-78b9-4c26-a422-d088a5b43a3d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:56.622394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9ac9ce-78b9-4c26-a422-d088a5b43a3d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:56.627498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e2599e-1f7d-442b-99b4-b3d14e49c3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:56.628386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2599e-1f7d-442b-99b4-b3d14e49c3af admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.628670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2599e-1f7d-442b-99b4-b3d14e49c3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.628947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e2599e-1f7d-442b-99b4-b3d14e49c3af admin admin] PATH_INFO: `/v3/projects/f1a347d4ae4c4aa4955e6c45033e9a48/users/c05e612ffe6f4f9c892d0223ec6411bf/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.645677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2599e-1f7d-442b-99b4-b3d14e49c3af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1a347d4ae4c4aa4955e6c45033e9a48, user_id=c05e612ffe6f4f9c892d0223ec6411bf, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:56.648388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e2599e-1f7d-442b-99b4-b3d14e49c3af admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:56.663078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1153/2308] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:26:56 2022] PUT /identity/v3/projects/f1a347d4ae4c4aa4955e6c45033e9a48/users/c05e612ffe6f4f9c892d0223ec6411bf/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:26:56.675766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfc33b-c543-4f25-bdda-8e6704b27513 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.676399 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfc33b-c543-4f25-bdda-8e6704b27513 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.677056 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcfc33b-c543-4f25-bdda-8e6704b27513 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.711032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7dcfc33b-c543-4f25-bdda-8e6704b27513 None None] MFA Rules not processed for user `c05e612ffe6f4f9c892d0223ec6411bf`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:56.712271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7dcfc33b-c543-4f25-bdda-8e6704b27513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:56.736646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cde412-1f84-4719-b1eb-64c3dba28485 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.736949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cde412-1f84-4719-b1eb-64c3dba28485 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.737202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33cde412-1f84-4719-b1eb-64c3dba28485 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.755348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-33cde412-1f84-4719-b1eb-64c3dba28485 None None] MFA Rules not processed for user `9192f2842ca044f08e5f95f98f2e1b88`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:26:56.756556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-33cde412-1f84-4719-b1eb-64c3dba28485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:56.799369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1156/2309] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:56 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:26:56.807657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfc33b-c543-4f25-bdda-8e6704b27513 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:26:56.816674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7dcfc33b-c543-4f25-bdda-8e6704b27513 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:26:56.824024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1154/2310] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:26:56 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:26:56.874364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcfc33b-c543-4f25-bdda-8e6704b27513 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:26:56.878017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b125ee-0f74-49b2-8934-40e35c5e885e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e46ec9c31094d03b6a413e09bc9f699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:26:56.878668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b125ee-0f74-49b2-8934-40e35c5e885e service glance] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:26:56.878954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b125ee-0f74-49b2-8934-40e35c5e885e service glance] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:26:56.879233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b125ee-0f74-49b2-8934-40e35c5e885e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:26:56.888420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b125ee-0f74-49b2-8934-40e35c5e885e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:26:56.890167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b125ee-0f74-49b2-8934-40e35c5e885e service glance] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:26:56.900094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1157/2311] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:26:56 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:27:06.816794 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=43,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:27:06.895787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:16.376330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cde412-1f84-4719-b1eb-64c3dba28485 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:16.377782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-33cde412-1f84-4719-b1eb-64c3dba28485 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:16.449295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33cde412-1f84-4719-b1eb-64c3dba28485 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:16.453983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:16.454864 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:16.455149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:16.455424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] PATH_INFO: `/v3/users/b0594cb7f3dc4574b022efcbbb97c288` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:16.456627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] RBAC: Authorizing `identity:delete_user(user_id=b0594cb7f3dc4574b022efcbbb97c288)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:16.459469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:16.509177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b0594cb7f3dc4574b022efcbbb97c288'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:16.521909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b0594cb7f3dc4574b022efcbbb97c288'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:16.528844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b0594cb7f3dc4574b022efcbbb97c288'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:16.529598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b0594cb7f3dc4574b022efcbbb97c288'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:16.530206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b0594cb7f3dc4574b022efcbbb97c288'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:16.543359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1155/2312] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:16 2022] DELETE /identity/v3/users/b0594cb7f3dc4574b022efcbbb97c288 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:16.548960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d40e78-a08c-4be2-affb-13f1304048c2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:16.548960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d40e78-a08c-4be2-affb-13f1304048c2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:16.548960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d40e78-a08c-4be2-affb-13f1304048c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:16.568660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f7d40e78-a08c-4be2-affb-13f1304048c2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:16.569929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f7d40e78-a08c-4be2-affb-13f1304048c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:16.633184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1158/2313] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:16.643031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:16.644421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:16.703770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffc603f-75f8-426a-96fa-b91563fe441b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:16.709621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f92e1d5-66d6-4538-9767-a60a7e4b2061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:16.710340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e1d5-66d6-4538-9767-a60a7e4b2061 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:16.710625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e1d5-66d6-4538-9767-a60a7e4b2061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:16.710979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f92e1d5-66d6-4538-9767-a60a7e4b2061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:16.721212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92e1d5-66d6-4538-9767-a60a7e4b2061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:16.722694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f92e1d5-66d6-4538-9767-a60a7e4b2061 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:16.733381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1156/2314] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:17.029818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d40e78-a08c-4be2-affb-13f1304048c2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.031253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f7d40e78-a08c-4be2-affb-13f1304048c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:17.043252 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d40e78-a08c-4be2-affb-13f1304048c2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.048479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.049393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.049723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.050000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] PATH_INFO: `/v3/projects/7be36032c7b5403682c9eb90a34d23a2` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.052081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7be36032c7b5403682c9eb90a34d23a2)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.054739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.105494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7be36032c7b5403682c9eb90a34d23a2', 'request_id': 'req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:17.112015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7be36032c7b5403682c9eb90a34d23a2', 'request_id': 'req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:17.118413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7be36032c7b5403682c9eb90a34d23a2', 'request_id': 'req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:17.132852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] The token cache is being invalidate because project 7be36032c7b5403682c9eb90a34d23a2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:17.133139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:17.133799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b5e38b-ca69-4145-8ae1-a1a5499dee31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:17.135632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1159/2315] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:17 2022] DELETE /identity/v3/projects/7be36032c7b5403682c9eb90a34d23a2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:17.140389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1882ea-6caa-450f-bfb3-395b542bc244 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.140680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1882ea-6caa-450f-bfb3-395b542bc244 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.140957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da1882ea-6caa-450f-bfb3-395b542bc244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.158524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-da1882ea-6caa-450f-bfb3-395b542bc244 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:17.159724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-da1882ea-6caa-450f-bfb3-395b542bc244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:17.178492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:17.227701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1157/2316] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:17.236514 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2baaf8-686f-45a6-bb2a-96f38a6d5c33 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.236686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2baaf8-686f-45a6-bb2a-96f38a6d5c33 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.236970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2baaf8-686f-45a6-bb2a-96f38a6d5c33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.256974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ba2baaf8-686f-45a6-bb2a-96f38a6d5c33 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:17.258069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ba2baaf8-686f-45a6-bb2a-96f38a6d5c33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:17.266152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1160/2317] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:17.270652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1882ea-6caa-450f-bfb3-395b542bc244 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.280938 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da1882ea-6caa-450f-bfb3-395b542bc244 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.287124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a7786-e9ef-441f-8912-3ed54e89c98d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.287878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a7786-e9ef-441f-8912-3ed54e89c98d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.288206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a7786-e9ef-441f-8912-3ed54e89c98d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.288559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a7786-e9ef-441f-8912-3ed54e89c98d admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.289292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a7786-e9ef-441f-8912-3ed54e89c98d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:17.289679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a7786-e9ef-441f-8912-3ed54e89c98d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.297052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a7786-e9ef-441f-8912-3ed54e89c98d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.311241 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1158/2318] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:28:17.318447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2baaf8-686f-45a6-bb2a-96f38a6d5c33 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.329014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2baaf8-686f-45a6-bb2a-96f38a6d5c33 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.334700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543cbaf-bbfb-4354-aa0b-f0e5da9509aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.335357 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543cbaf-bbfb-4354-aa0b-f0e5da9509aa admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.335656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543cbaf-bbfb-4354-aa0b-f0e5da9509aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.335932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543cbaf-bbfb-4354-aa0b-f0e5da9509aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.338310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543cbaf-bbfb-4354-aa0b-f0e5da9509aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.340494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543cbaf-bbfb-4354-aa0b-f0e5da9509aa admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.367317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1161/2319] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:17 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:17.371414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a7786-e9ef-441f-8912-3ed54e89c98d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.381362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a7786-e9ef-441f-8912-3ed54e89c98d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.438048 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802da950-014e-455c-85a0-842d0e1d5f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.438721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802da950-014e-455c-85a0-842d0e1d5f1d admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.439006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802da950-014e-455c-85a0-842d0e1d5f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.439282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802da950-014e-455c-85a0-842d0e1d5f1d admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.440081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802da950-014e-455c-85a0-842d0e1d5f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.442461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802da950-014e-455c-85a0-842d0e1d5f1d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.462600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1159/2320] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:17 2022] POST /identity/v3/users => generated 289 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:17.467345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543cbaf-bbfb-4354-aa0b-f0e5da9509aa admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.479269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543cbaf-bbfb-4354-aa0b-f0e5da9509aa admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.483967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb7901-c573-47fe-b650-fa123d480f67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.484630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb7901-c573-47fe-b650-fa123d480f67 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.484913 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb7901-c573-47fe-b650-fa123d480f67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.485190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffb7901-c573-47fe-b650-fa123d480f67 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.485947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb7901-c573-47fe-b650-fa123d480f67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.487678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffb7901-c573-47fe-b650-fa123d480f67 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.499378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1162/2321] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:17.503724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802da950-014e-455c-85a0-842d0e1d5f1d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.516703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802da950-014e-455c-85a0-842d0e1d5f1d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.522267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bad5f8a5-8be6-4bdb-9128-6014cff324f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.522933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5f8a5-8be6-4bdb-9128-6014cff324f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.523221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5f8a5-8be6-4bdb-9128-6014cff324f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.523499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bad5f8a5-8be6-4bdb-9128-6014cff324f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.524278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5f8a5-8be6-4bdb-9128-6014cff324f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.525944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bad5f8a5-8be6-4bdb-9128-6014cff324f1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.536601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1160/2322] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:17 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:17.541535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb7901-c573-47fe-b650-fa123d480f67 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.553563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffb7901-c573-47fe-b650-fa123d480f67 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.558320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb3e50-57ce-4c4b-b030-0df6fc0ab5ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.559202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb3e50-57ce-4c4b-b030-0df6fc0ab5ac admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.559484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb3e50-57ce-4c4b-b030-0df6fc0ab5ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.559764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacb3e50-57ce-4c4b-b030-0df6fc0ab5ac admin admin] PATH_INFO: `/v3/projects/8064950f9a1942149aa8d6c08c206850/users/8c67fe170c60476d88610d85f2528c52/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.579103 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb3e50-57ce-4c4b-b030-0df6fc0ab5ac admin admin] RBAC: Authorizing `identity:create_grant(project_id=8064950f9a1942149aa8d6c08c206850, user_id=8c67fe170c60476d88610d85f2528c52, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.581802 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bacb3e50-57ce-4c4b-b030-0df6fc0ab5ac admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.595158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1163/2323] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:17 2022] PUT /identity/v3/projects/8064950f9a1942149aa8d6c08c206850/users/8c67fe170c60476d88610d85f2528c52/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:17.600516 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec38e-3a2d-48fa-ab5d-58fc450b3663 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.600804 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec38e-3a2d-48fa-ab5d-58fc450b3663 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.601091 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-028ec38e-3a2d-48fa-ab5d-58fc450b3663 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.619589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-028ec38e-3a2d-48fa-ab5d-58fc450b3663 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:17.620730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-028ec38e-3a2d-48fa-ab5d-58fc450b3663 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:17.680529 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1161/2324] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:17.691430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb3e50-57ce-4c4b-b030-0df6fc0ab5ac admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:17.693836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bacb3e50-57ce-4c4b-b030-0df6fc0ab5ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:17.717822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:17.756532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bacb3e50-57ce-4c4b-b030-0df6fc0ab5ac admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:17.760146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a4d4f-e1da-4fc8-b65c-5057648861e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:17.760786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a4d4f-e1da-4fc8-b65c-5057648861e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:17.761067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a4d4f-e1da-4fc8-b65c-5057648861e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:17.761339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f89a4d4f-e1da-4fc8-b65c-5057648861e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:17.770600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a4d4f-e1da-4fc8-b65c-5057648861e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:17.772021 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f89a4d4f-e1da-4fc8-b65c-5057648861e2 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:17.781653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1164/2325] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:21.883953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae9966-fe7e-4afa-9504-12764deeda26 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:21.884445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae9966-fe7e-4afa-9504-12764deeda26 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:21.884445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ae9966-fe7e-4afa-9504-12764deeda26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:21.913005 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-35ae9966-fe7e-4afa-9504-12764deeda26 None None] MFA Rules not processed for user `8c67fe170c60476d88610d85f2528c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:21.914258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-35ae9966-fe7e-4afa-9504-12764deeda26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:21.930913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:21.986378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1162/2326] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:21.991008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a4d4f-e1da-4fc8-b65c-5057648861e2 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:22.000752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f89a4d4f-e1da-4fc8-b65c-5057648861e2 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:22.007271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b61a9-8c62-45e6-9455-51443c66b8a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:22.007941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b61a9-8c62-45e6-9455-51443c66b8a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.008220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b61a9-8c62-45e6-9455-51443c66b8a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.008663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b6b61a9-8c62-45e6-9455-51443c66b8a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.009585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b61a9-8c62-45e6-9455-51443c66b8a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:22.011448 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b6b61a9-8c62-45e6-9455-51443c66b8a3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.040564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1165/2327] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:21 2022] POST /identity/v3/projects => generated 379 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:22.044654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae9966-fe7e-4afa-9504-12764deeda26 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:22.056552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35ae9966-fe7e-4afa-9504-12764deeda26 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:22.124098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c767e-e57b-4d46-a7fa-46e8e606989f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:22.124820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c767e-e57b-4d46-a7fa-46e8e606989f admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.125128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c767e-e57b-4d46-a7fa-46e8e606989f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.125399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3c767e-e57b-4d46-a7fa-46e8e606989f admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.126109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c767e-e57b-4d46-a7fa-46e8e606989f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:22.128030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3c767e-e57b-4d46-a7fa-46e8e606989f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.156063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1163/2328] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:22 2022] POST /identity/v3/users => generated 296 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:22.161408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b61a9-8c62-45e6-9455-51443c66b8a3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:22.174857 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b6b61a9-8c62-45e6-9455-51443c66b8a3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:22.180290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c939b87-22d9-4ccc-a320-363e64c3ecbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:22.180971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939b87-22d9-4ccc-a320-363e64c3ecbb admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.181261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939b87-22d9-4ccc-a320-363e64c3ecbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.181589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c939b87-22d9-4ccc-a320-363e64c3ecbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.182322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c939b87-22d9-4ccc-a320-363e64c3ecbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:22.184354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c939b87-22d9-4ccc-a320-363e64c3ecbb admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.200735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1166/2329] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:22.201862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c767e-e57b-4d46-a7fa-46e8e606989f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:22.214182 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3c767e-e57b-4d46-a7fa-46e8e606989f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:22.219353 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8523e-32f5-4d1d-917e-002a50e04d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:22.220604 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8523e-32f5-4d1d-917e-002a50e04d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.220739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8523e-32f5-4d1d-917e-002a50e04d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.220945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8523e-32f5-4d1d-917e-002a50e04d2e admin admin] PATH_INFO: `/v3/projects/466bd0dde0234ba78083c98206c4f462/users/312a873d649e40fd9ec1949e4d077090/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.240387 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8523e-32f5-4d1d-917e-002a50e04d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=466bd0dde0234ba78083c98206c4f462, user_id=312a873d649e40fd9ec1949e4d077090, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:22.243116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8523e-32f5-4d1d-917e-002a50e04d2e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.257080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1164/2330] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:22 2022] PUT /identity/v3/projects/466bd0dde0234ba78083c98206c4f462/users/312a873d649e40fd9ec1949e4d077090/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:28:22.261357 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c939b87-22d9-4ccc-a320-363e64c3ecbb admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:22.272301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c939b87-22d9-4ccc-a320-363e64c3ecbb admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:22.343081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c406502-359c-4916-b9b0-45cc33060e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:22.343945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c406502-359c-4916-b9b0-45cc33060e1b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.344341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c406502-359c-4916-b9b0-45cc33060e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.344468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c406502-359c-4916-b9b0-45cc33060e1b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.345270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c406502-359c-4916-b9b0-45cc33060e1b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:22.352452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c406502-359c-4916-b9b0-45cc33060e1b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.367032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1167/2331] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:22 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:22.377878 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8523e-32f5-4d1d-917e-002a50e04d2e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:22.389009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8523e-32f5-4d1d-917e-002a50e04d2e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:22.395246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef6454-bbb8-4e36-8349-e996c28c28c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:22.396159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef6454-bbb8-4e36-8349-e996c28c28c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:22.396443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef6454-bbb8-4e36-8349-e996c28c28c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:22.396721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bef6454-bbb8-4e36-8349-e996c28c28c0 admin admin] PATH_INFO: `/v3/projects/466bd0dde0234ba78083c98206c4f462/users/312a873d649e40fd9ec1949e4d077090/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:22.398763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef6454-bbb8-4e36-8349-e996c28c28c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=466bd0dde0234ba78083c98206c4f462, user_id=312a873d649e40fd9ec1949e4d077090, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:22.401432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bef6454-bbb8-4e36-8349-e996c28c28c0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:22.413759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1165/2332] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:22 2022] PUT /identity/v3/projects/466bd0dde0234ba78083c98206c4f462/users/312a873d649e40fd9ec1949e4d077090/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:27.069793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39603963-a332-425f-86ac-8ddcb6a66fa3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:27.070817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39603963-a332-425f-86ac-8ddcb6a66fa3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:27.071401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39603963-a332-425f-86ac-8ddcb6a66fa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:27.106356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-39603963-a332-425f-86ac-8ddcb6a66fa3 None None] MFA Rules not processed for user `312a873d649e40fd9ec1949e4d077090`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:27.107684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-39603963-a332-425f-86ac-8ddcb6a66fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:27.193912 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1168/2333] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:27 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:27.208326 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef6454-bbb8-4e36-8349-e996c28c28c0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:27.212449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9bef6454-bbb8-4e36-8349-e996c28c28c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:27.294592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bef6454-bbb8-4e36-8349-e996c28c28c0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:27.300101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21364ffa-8557-4871-94ad-71cfae21896a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:27.301302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364ffa-8557-4871-94ad-71cfae21896a service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:27.301622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364ffa-8557-4871-94ad-71cfae21896a service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:27.301903 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21364ffa-8557-4871-94ad-71cfae21896a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:27.307525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:27.384646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21364ffa-8557-4871-94ad-71cfae21896a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:27.386324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21364ffa-8557-4871-94ad-71cfae21896a service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:27.396638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1166/2334] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:37.190705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:37.391623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=24 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:40.883082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39603963-a332-425f-86ac-8ddcb6a66fa3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:40.884512 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-39603963-a332-425f-86ac-8ddcb6a66fa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:40.952697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39603963-a332-425f-86ac-8ddcb6a66fa3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:40.957796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:40.958698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:40.958964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:40.959239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] PATH_INFO: `/v3/users/c05e612ffe6f4f9c892d0223ec6411bf` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:40.960443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c05e612ffe6f4f9c892d0223ec6411bf)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:40.962429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:41.011653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c05e612ffe6f4f9c892d0223ec6411bf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.024043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c05e612ffe6f4f9c892d0223ec6411bf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.031015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c05e612ffe6f4f9c892d0223ec6411bf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.031685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c05e612ffe6f4f9c892d0223ec6411bf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.032298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c05e612ffe6f4f9c892d0223ec6411bf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.046199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1169/2335] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:28:40 2022] DELETE /identity/v3/users/c05e612ffe6f4f9c892d0223ec6411bf => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:28:41.051736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ebda0-e214-49e1-ae38-806fdca312a4 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.052254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ebda0-e214-49e1-ae38-806fdca312a4 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.052254 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ebda0-e214-49e1-ae38-806fdca312a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.070622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-064ebda0-e214-49e1-ae38-806fdca312a4 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:41.071831 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-064ebda0-e214-49e1-ae38-806fdca312a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:41.134182 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1167/2336] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:28:41.144810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:41.146630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:41.232333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92353ec3-cfde-47c0-b3f4-b0c1ad44d461 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:41.236611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f339a83d-201e-4ee6-ba95-2b8f24ead576 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:41.237267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339a83d-201e-4ee6-ba95-2b8f24ead576 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.238235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339a83d-201e-4ee6-ba95-2b8f24ead576 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.238461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f339a83d-201e-4ee6-ba95-2b8f24ead576 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.249474 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339a83d-201e-4ee6-ba95-2b8f24ead576 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:41.251158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f339a83d-201e-4ee6-ba95-2b8f24ead576 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:41.264138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1170/2337] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:28:41.563931 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ebda0-e214-49e1-ae38-806fdca312a4 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:41.566317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-064ebda0-e214-49e1-ae38-806fdca312a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:41.580027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ebda0-e214-49e1-ae38-806fdca312a4 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:41.584956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:41.585872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.586158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.586625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] PATH_INFO: `/v3/projects/f1a347d4ae4c4aa4955e6c45033e9a48` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.588103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1a347d4ae4c4aa4955e6c45033e9a48)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:41.590842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:41.641647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1a347d4ae4c4aa4955e6c45033e9a48', 'request_id': 'req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.647405 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1a347d4ae4c4aa4955e6c45033e9a48', 'request_id': 'req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.655128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1a347d4ae4c4aa4955e6c45033e9a48', 'request_id': 'req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.667012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] The token cache is being invalidate because project f1a347d4ae4c4aa4955e6c45033e9a48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:28:41.667012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.667698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f16fc62c-c0f1-412b-897b-fd7ace8bb7f7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:28:41.669828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1168/2338] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:28:41 2022] DELETE /identity/v3/projects/f1a347d4ae4c4aa4955e6c45033e9a48 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:41.675699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657bbdf-5ede-4117-82a4-e74999258699 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.675699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657bbdf-5ede-4117-82a4-e74999258699 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.676605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657bbdf-5ede-4117-82a4-e74999258699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.695195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5657bbdf-5ede-4117-82a4-e74999258699 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:41.696403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5657bbdf-5ede-4117-82a4-e74999258699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:41.713179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:41.760753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1171/2339] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:41.768354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ed6f7-5655-47cb-b416-acccae88b776 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.768787 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ed6f7-5655-47cb-b416-acccae88b776 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.769063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942ed6f7-5655-47cb-b416-acccae88b776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.788159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-942ed6f7-5655-47cb-b416-acccae88b776 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:41.789382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-942ed6f7-5655-47cb-b416-acccae88b776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:41.798645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1169/2340] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 19:28:41.803488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657bbdf-5ede-4117-82a4-e74999258699 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:41.813477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657bbdf-5ede-4117-82a4-e74999258699 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:41.818962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265e67a-f7de-4701-9ceb-3f40ce59c416 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:41.819643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265e67a-f7de-4701-9ceb-3f40ce59c416 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.819925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265e67a-f7de-4701-9ceb-3f40ce59c416 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.820200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4265e67a-f7de-4701-9ceb-3f40ce59c416 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.820912 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265e67a-f7de-4701-9ceb-3f40ce59c416 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 19:28:41.821219 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265e67a-f7de-4701-9ceb-3f40ce59c416 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:41.822861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4265e67a-f7de-4701-9ceb-3f40ce59c416 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:41.835275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1172/2341] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 19:28:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 19:28:41.839988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ed6f7-5655-47cb-b416-acccae88b776 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:41.851639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942ed6f7-5655-47cb-b416-acccae88b776 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:41.856444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d485267-04cc-4ed7-b3e3-bda670a4c0f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:41.857496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d485267-04cc-4ed7-b3e3-bda670a4c0f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.857861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d485267-04cc-4ed7-b3e3-bda670a4c0f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.858141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d485267-04cc-4ed7-b3e3-bda670a4c0f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.858854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d485267-04cc-4ed7-b3e3-bda670a4c0f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:41.860813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d485267-04cc-4ed7-b3e3-bda670a4c0f3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:41.889350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1170/2342] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:41 2022] POST /identity/v3/projects => generated 373 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:28:41.894393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265e67a-f7de-4701-9ceb-3f40ce59c416 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:41.904657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4265e67a-f7de-4701-9ceb-3f40ce59c416 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:41.971159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63adfcc9-3d30-4be6-8526-b1bae199a39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:41.971895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adfcc9-3d30-4be6-8526-b1bae199a39f admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:41.972172 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adfcc9-3d30-4be6-8526-b1bae199a39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:41.972447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63adfcc9-3d30-4be6-8526-b1bae199a39f admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:41.973148 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63adfcc9-3d30-4be6-8526-b1bae199a39f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:41.975193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63adfcc9-3d30-4be6-8526-b1bae199a39f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:41.996407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1173/2343] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:41 2022] POST /identity/v3/users => generated 287 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 19:28:42.001917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d485267-04cc-4ed7-b3e3-bda670a4c0f3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.015816 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d485267-04cc-4ed7-b3e3-bda670a4c0f3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.021453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1014cc3-c35e-4de0-ac39-78cd9ab47d1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.022157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1014cc3-c35e-4de0-ac39-78cd9ab47d1c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.022440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1014cc3-c35e-4de0-ac39-78cd9ab47d1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.022719 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1014cc3-c35e-4de0-ac39-78cd9ab47d1c admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.023456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1014cc3-c35e-4de0-ac39-78cd9ab47d1c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.025078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1014cc3-c35e-4de0-ac39-78cd9ab47d1c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.035971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1171/2344] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:42.041287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63adfcc9-3d30-4be6-8526-b1bae199a39f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.053203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63adfcc9-3d30-4be6-8526-b1bae199a39f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.058547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02a3dd3-bea8-430c-b2fe-129c78ba8b75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.059236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a3dd3-bea8-430c-b2fe-129c78ba8b75 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.059519 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a3dd3-bea8-430c-b2fe-129c78ba8b75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.059801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02a3dd3-bea8-430c-b2fe-129c78ba8b75 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.060550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a3dd3-bea8-430c-b2fe-129c78ba8b75 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.062221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02a3dd3-bea8-430c-b2fe-129c78ba8b75 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.074417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1174/2345] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:42 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:42.078736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1014cc3-c35e-4de0-ac39-78cd9ab47d1c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.092210 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1014cc3-c35e-4de0-ac39-78cd9ab47d1c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.097321 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291897ed-90e4-4212-b4fe-c58138f9ebd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.098250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291897ed-90e4-4212-b4fe-c58138f9ebd4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.098534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291897ed-90e4-4212-b4fe-c58138f9ebd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.098814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291897ed-90e4-4212-b4fe-c58138f9ebd4 admin admin] PATH_INFO: `/v3/projects/da296fe635c84b99b1645ceaf0138b83/users/4d7204f863c64eeeb7dab59a30c447b8/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.114319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291897ed-90e4-4212-b4fe-c58138f9ebd4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da296fe635c84b99b1645ceaf0138b83, user_id=4d7204f863c64eeeb7dab59a30c447b8, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.117116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291897ed-90e4-4212-b4fe-c58138f9ebd4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.132656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1172/2346] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:42 2022] PUT /identity/v3/projects/da296fe635c84b99b1645ceaf0138b83/users/4d7204f863c64eeeb7dab59a30c447b8/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:42.138288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b39ef-ee0b-414b-b71c-ac25602a04d6 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.138708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b39ef-ee0b-414b-b71c-ac25602a04d6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.138993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286b39ef-ee0b-414b-b71c-ac25602a04d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.158845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-286b39ef-ee0b-414b-b71c-ac25602a04d6 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:42.160342 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-286b39ef-ee0b-414b-b71c-ac25602a04d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:42.235653 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1175/2347] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:42.245684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291897ed-90e4-4212-b4fe-c58138f9ebd4 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:42.247425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-291897ed-90e4-4212-b4fe-c58138f9ebd4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:42.277532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:42.317415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291897ed-90e4-4212-b4fe-c58138f9ebd4 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:42.321413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc316825-f959-4d81-8042-0352586ef800 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:42.322144 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc316825-f959-4d81-8042-0352586ef800 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:42.322431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc316825-f959-4d81-8042-0352586ef800 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:42.322711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc316825-f959-4d81-8042-0352586ef800 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:42.336951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc316825-f959-4d81-8042-0352586ef800 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:42.338693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc316825-f959-4d81-8042-0352586ef800 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:42.351840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1173/2348] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:28:43.522827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b39ef-ee0b-414b-b71c-ac25602a04d6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:43.524326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-286b39ef-ee0b-414b-b71c-ac25602a04d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:43.568510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:43.601361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286b39ef-ee0b-414b-b71c-ac25602a04d6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:43.606422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66550e03-5df5-4b1d-82d0-ac8808fd4f50 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:43.607251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66550e03-5df5-4b1d-82d0-ac8808fd4f50 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:43.607636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66550e03-5df5-4b1d-82d0-ac8808fd4f50 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:43.607996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66550e03-5df5-4b1d-82d0-ac8808fd4f50 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:43.610673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-66550e03-5df5-4b1d-82d0-ac8808fd4f50 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:43.730643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66550e03-5df5-4b1d-82d0-ac8808fd4f50 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:43.732306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66550e03-5df5-4b1d-82d0-ac8808fd4f50 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:43.763540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1176/2349] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:46.852046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deac52-5c01-4482-8a99-1454f67e442e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:46.852656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deac52-5c01-4482-8a99-1454f67e442e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:46.852656 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10deac52-5c01-4482-8a99-1454f67e442e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:46.905000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-10deac52-5c01-4482-8a99-1454f67e442e None None] MFA Rules not processed for user `4d7204f863c64eeeb7dab59a30c447b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:46.906288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-10deac52-5c01-4482-8a99-1454f67e442e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:47.002441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1174/2350] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:46 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:47.007734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66550e03-5df5-4b1d-82d0-ac8808fd4f50 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:47.022202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66550e03-5df5-4b1d-82d0-ac8808fd4f50 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:47.029278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034d68d-96fe-4958-9218-71d173bcfdb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:47.030236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d68d-96fe-4958-9218-71d173bcfdb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:47.030564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d68d-96fe-4958-9218-71d173bcfdb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:47.030931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034d68d-96fe-4958-9218-71d173bcfdb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:47.032213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034d68d-96fe-4958-9218-71d173bcfdb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:47.035958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034d68d-96fe-4958-9218-71d173bcfdb4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:47.073820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1177/2351] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 19:28:47 2022] POST /identity/v3/projects => generated 369 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:47.078926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deac52-5c01-4482-8a99-1454f67e442e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:47.097206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10deac52-5c01-4482-8a99-1454f67e442e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:47.166442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c8e38-c958-4e1d-b306-efafe048e4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:47.167126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c8e38-c958-4e1d-b306-efafe048e4cc admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:47.167410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c8e38-c958-4e1d-b306-efafe048e4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:47.167685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421c8e38-c958-4e1d-b306-efafe048e4cc admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:47.168377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c8e38-c958-4e1d-b306-efafe048e4cc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:47.170424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421c8e38-c958-4e1d-b306-efafe048e4cc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:47.198440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1175/2352] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 19:28:47 2022] POST /identity/v3/users => generated 291 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 19:28:47.202945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034d68d-96fe-4958-9218-71d173bcfdb4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:47.214511 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034d68d-96fe-4958-9218-71d173bcfdb4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:47.219281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be50a0-e73a-42d2-8c58-9ba53c9cfe04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:47.219959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be50a0-e73a-42d2-8c58-9ba53c9cfe04 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:47.220245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be50a0-e73a-42d2-8c58-9ba53c9cfe04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:47.220524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be50a0-e73a-42d2-8c58-9ba53c9cfe04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:47.221256 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be50a0-e73a-42d2-8c58-9ba53c9cfe04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:47.223443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be50a0-e73a-42d2-8c58-9ba53c9cfe04 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:47.235281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1178/2353] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:47.240288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c8e38-c958-4e1d-b306-efafe048e4cc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:47.250922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421c8e38-c958-4e1d-b306-efafe048e4cc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:47.256451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b839f3-93e6-44f7-9f60-9d8aaad0662c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:47.257367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b839f3-93e6-44f7-9f60-9d8aaad0662c admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:47.257686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b839f3-93e6-44f7-9f60-9d8aaad0662c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:47.257971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b839f3-93e6-44f7-9f60-9d8aaad0662c admin admin] PATH_INFO: `/v3/projects/dc3dc6e75b914c7c9cd8a664d7431d90/users/166b54d7b99a432e931a10e2c1b71eb0/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:47.279955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b839f3-93e6-44f7-9f60-9d8aaad0662c admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3dc6e75b914c7c9cd8a664d7431d90, user_id=166b54d7b99a432e931a10e2c1b71eb0, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:47.282630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b839f3-93e6-44f7-9f60-9d8aaad0662c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:47.308731 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1176/2354] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:47 2022] PUT /identity/v3/projects/dc3dc6e75b914c7c9cd8a664d7431d90/users/166b54d7b99a432e931a10e2c1b71eb0/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:28:47.312665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be50a0-e73a-42d2-8c58-9ba53c9cfe04 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:47.324160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be50a0-e73a-42d2-8c58-9ba53c9cfe04 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:47.388048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820ccfe-a901-4a4c-88b8-55db7bfb1e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:47.388733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820ccfe-a901-4a4c-88b8-55db7bfb1e4c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:47.389017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820ccfe-a901-4a4c-88b8-55db7bfb1e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:47.389300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d820ccfe-a901-4a4c-88b8-55db7bfb1e4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:47.390067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820ccfe-a901-4a4c-88b8-55db7bfb1e4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:47.391709 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d820ccfe-a901-4a4c-88b8-55db7bfb1e4c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:47.402776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1179/2355] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 19:28:47 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 19:28:47.407178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b839f3-93e6-44f7-9f60-9d8aaad0662c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:47.418950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b839f3-93e6-44f7-9f60-9d8aaad0662c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:47.425453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b9a07-debc-4d34-95cf-fd7987e7037b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:47.426653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b9a07-debc-4d34-95cf-fd7987e7037b admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:47.427002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b9a07-debc-4d34-95cf-fd7987e7037b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:47.427349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b9a07-debc-4d34-95cf-fd7987e7037b admin admin] PATH_INFO: `/v3/projects/dc3dc6e75b914c7c9cd8a664d7431d90/users/166b54d7b99a432e931a10e2c1b71eb0/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:47.429600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b9a07-debc-4d34-95cf-fd7987e7037b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc3dc6e75b914c7c9cd8a664d7431d90, user_id=166b54d7b99a432e931a10e2c1b71eb0, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:47.432721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6b9a07-debc-4d34-95cf-fd7987e7037b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:47.446934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1177/2356] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 19:28:47 2022] PUT /identity/v3/projects/dc3dc6e75b914c7c9cd8a664d7431d90/users/166b54d7b99a432e931a10e2c1b71eb0/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:28:47.740074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820ccfe-a901-4a4c-88b8-55db7bfb1e4c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:47.742718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d820ccfe-a901-4a4c-88b8-55db7bfb1e4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:47.785973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:47.824784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d820ccfe-a901-4a4c-88b8-55db7bfb1e4c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:47.890354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26281bc-4584-4f6a-be68-3a47f70e93d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:47.891720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26281bc-4584-4f6a-be68-3a47f70e93d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:47.892010 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26281bc-4584-4f6a-be68-3a47f70e93d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:47.892311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f26281bc-4584-4f6a-be68-3a47f70e93d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:47.896029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f26281bc-4584-4f6a-be68-3a47f70e93d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:48.018135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26281bc-4584-4f6a-be68-3a47f70e93d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:48.020330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f26281bc-4584-4f6a-be68-3a47f70e93d4 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:48.037842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1180/2357] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:53.043463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d544525-49b1-4ffa-8458-b45bbf615797 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:53.044007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d544525-49b1-4ffa-8458-b45bbf615797 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:53.044007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d544525-49b1-4ffa-8458-b45bbf615797 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:53.051674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:28:53.075789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6d544525-49b1-4ffa-8458-b45bbf615797 None None] MFA Rules not processed for user `166b54d7b99a432e931a10e2c1b71eb0`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:53.077227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6d544525-49b1-4ffa-8458-b45bbf615797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:53.158123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1178/2358] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:53 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:53.172676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26281bc-4584-4f6a-be68-3a47f70e93d4 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:53.174183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f26281bc-4584-4f6a-be68-3a47f70e93d4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:53.188720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f26281bc-4584-4f6a-be68-3a47f70e93d4 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:53.194608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d58dda-7001-4c1f-9db2-c9d574717b7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:53.195506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d58dda-7001-4c1f-9db2-c9d574717b7c service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:53.195994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d58dda-7001-4c1f-9db2-c9d574717b7c service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:53.196544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d58dda-7001-4c1f-9db2-c9d574717b7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:53.267862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d58dda-7001-4c1f-9db2-c9d574717b7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:53.270591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d58dda-7001-4c1f-9db2-c9d574717b7c service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:53.283803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1181/2359] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:53 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:54.366032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d544525-49b1-4ffa-8458-b45bbf615797 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:54.367916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6d544525-49b1-4ffa-8458-b45bbf615797 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:54.390120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d544525-49b1-4ffa-8458-b45bbf615797 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:54.398593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095317f5-c171-4d1e-af09-7a501c27d953 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:54.399470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095317f5-c171-4d1e-af09-7a501c27d953 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:54.399949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095317f5-c171-4d1e-af09-7a501c27d953 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:54.400320 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095317f5-c171-4d1e-af09-7a501c27d953 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:54.402727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-095317f5-c171-4d1e-af09-7a501c27d953 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:54.477895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095317f5-c171-4d1e-af09-7a501c27d953 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:54.479680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095317f5-c171-4d1e-af09-7a501c27d953 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:54.491817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1179/2360] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:28:54.513704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c256a-1b6c-43bd-a85a-47ddc4a759ec None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:54.514195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c256a-1b6c-43bd-a85a-47ddc4a759ec None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:54.514300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c256a-1b6c-43bd-a85a-47ddc4a759ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:54.537139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7e1c256a-1b6c-43bd-a85a-47ddc4a759ec None None] MFA Rules not processed for user `4d7204f863c64eeeb7dab59a30c447b8`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:28:54.539077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7e1c256a-1b6c-43bd-a85a-47ddc4a759ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:54.549229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1182/2361] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:28:54 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:28:57.308874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095317f5-c171-4d1e-af09-7a501c27d953 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:28:57.323744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095317f5-c171-4d1e-af09-7a501c27d953 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:28:57.329323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab6018-0558-4ec5-a2e1-bb9baabc9d97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:28:57.330004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab6018-0558-4ec5-a2e1-bb9baabc9d97 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:28:57.330288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab6018-0558-4ec5-a2e1-bb9baabc9d97 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:28:57.331240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ab6018-0558-4ec5-a2e1-bb9baabc9d97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:28:57.333638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-16ab6018-0558-4ec5-a2e1-bb9baabc9d97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:28:57.399929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab6018-0558-4ec5-a2e1-bb9baabc9d97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:28:57.401554 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ab6018-0558-4ec5-a2e1-bb9baabc9d97 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:28:57.410884 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1180/2362] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:28:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:04.531013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:07.406366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=88 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:10.113003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f8524-7669-4175-96b3-29fcea2a739a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.113566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f8524-7669-4175-96b3-29fcea2a739a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.113697 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-602f8524-7669-4175-96b3-29fcea2a739a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.131086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-602f8524-7669-4175-96b3-29fcea2a739a None None] MFA Rules not processed for user `78ae826a812b44328c48b82e9498ad8b`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:10.132280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-602f8524-7669-4175-96b3-29fcea2a739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:10.189799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1183/2363] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:29:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:10.431031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab6018-0558-4ec5-a2e1-bb9baabc9d97 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.440552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ab6018-0558-4ec5-a2e1-bb9baabc9d97 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:10.444433 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f7dc89-f049-45fd-9235-850031f0060a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:10.445121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7dc89-f049-45fd-9235-850031f0060a service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.445406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7dc89-f049-45fd-9235-850031f0060a service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.445722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f7dc89-f049-45fd-9235-850031f0060a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.447557 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-32f7dc89-f049-45fd-9235-850031f0060a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:10.457919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7dc89-f049-45fd-9235-850031f0060a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:10.459388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32f7dc89-f049-45fd-9235-850031f0060a service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:10.468211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1181/2364] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:29:10.815348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f8524-7669-4175-96b3-29fcea2a739a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:10.817000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-602f8524-7669-4175-96b3-29fcea2a739a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:10.901756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-602f8524-7669-4175-96b3-29fcea2a739a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:10.907438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1b84b-0ce2-4bb2-9526-3dd691d3eaa4 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:10.908088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1b84b-0ce2-4bb2-9526-3dd691d3eaa4 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:10.908372 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1b84b-0ce2-4bb2-9526-3dd691d3eaa4 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:10.908647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec1b84b-0ce2-4bb2-9526-3dd691d3eaa4 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:10.910431 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cec1b84b-0ce2-4bb2-9526-3dd691d3eaa4 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:10.922612 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1b84b-0ce2-4bb2-9526-3dd691d3eaa4 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:10.924197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec1b84b-0ce2-4bb2-9526-3dd691d3eaa4 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:10.934875 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1184/2365] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:12.395626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8f07a-db6d-41e6-a736-d1ae4310d16d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:12.396165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8f07a-db6d-41e6-a736-d1ae4310d16d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:12.396351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8f07a-db6d-41e6-a736-d1ae4310d16d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:12.414870 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b4f8f07a-db6d-41e6-a736-d1ae4310d16d None None] MFA Rules not processed for user `8c67fe170c60476d88610d85f2528c52`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:12.416329 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b4f8f07a-db6d-41e6-a736-d1ae4310d16d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:12.479929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1182/2366] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 19:29:12 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:12.853022 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1b84b-0ce2-4bb2-9526-3dd691d3eaa4 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:12.866762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec1b84b-0ce2-4bb2-9526-3dd691d3eaa4 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:12.870405 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566b52a-18be-470e-9c87-76b7708e9339 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:12.871087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566b52a-18be-470e-9c87-76b7708e9339 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:12.871371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566b52a-18be-470e-9c87-76b7708e9339 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:12.871728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2566b52a-18be-470e-9c87-76b7708e9339 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:12.874784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2566b52a-18be-470e-9c87-76b7708e9339 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:12.885781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566b52a-18be-470e-9c87-76b7708e9339 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:12.887387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2566b52a-18be-470e-9c87-76b7708e9339 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:12.896828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1185/2367] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:12 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:13.770238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8f07a-db6d-41e6-a736-d1ae4310d16d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:13.784918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8f07a-db6d-41e6-a736-d1ae4310d16d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:13.791951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1875ad-09e9-46d7-b206-41d6a8a8642a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:13.792622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1875ad-09e9-46d7-b206-41d6a8a8642a service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:13.792909 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1875ad-09e9-46d7-b206-41d6a8a8642a service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:13.793185 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd1875ad-09e9-46d7-b206-41d6a8a8642a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:13.795642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fd1875ad-09e9-46d7-b206-41d6a8a8642a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:13.808429 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1875ad-09e9-46d7-b206-41d6a8a8642a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:13.810075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd1875ad-09e9-46d7-b206-41d6a8a8642a service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:13.820854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1183/2368] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:22.892868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=63 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:23.816334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=37 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:26.330804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566b52a-18be-470e-9c87-76b7708e9339 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:26.346807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2566b52a-18be-470e-9c87-76b7708e9339 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:26.353185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd82dd-796d-4787-8464-a987382deae7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:26.354264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd82dd-796d-4787-8464-a987382deae7 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:26.354708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd82dd-796d-4787-8464-a987382deae7 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:26.355141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcd82dd-796d-4787-8464-a987382deae7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:26.357859 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7dcd82dd-796d-4787-8464-a987382deae7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:26.371268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd82dd-796d-4787-8464-a987382deae7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:26.372887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dcd82dd-796d-4787-8464-a987382deae7 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:26.384908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1186/2369] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:34.359479 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1875ad-09e9-46d7-b206-41d6a8a8642a service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:34.370068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd1875ad-09e9-46d7-b206-41d6a8a8642a service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:34.373599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de0d310-c628-453b-9bc1-0c6c63199e9f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:34.374246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0d310-c628-453b-9bc1-0c6c63199e9f service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:34.374525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0d310-c628-453b-9bc1-0c6c63199e9f service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:34.374799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de0d310-c628-453b-9bc1-0c6c63199e9f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:34.376872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3de0d310-c628-453b-9bc1-0c6c63199e9f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:34.386792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0d310-c628-453b-9bc1-0c6c63199e9f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:34.388216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de0d310-c628-453b-9bc1-0c6c63199e9f service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:34.397125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1184/2370] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:34 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:36.379806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:43.422925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd82dd-796d-4787-8464-a987382deae7 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:43.432409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dcd82dd-796d-4787-8464-a987382deae7 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:43.436125 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338b0f8-f2e4-4427-b61b-1648b92a2129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:43.436759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338b0f8-f2e4-4427-b61b-1648b92a2129 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:43.437042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338b0f8-f2e4-4427-b61b-1648b92a2129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:43.437844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1338b0f8-f2e4-4427-b61b-1648b92a2129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:43.439663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1338b0f8-f2e4-4427-b61b-1648b92a2129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:43.507855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338b0f8-f2e4-4427-b61b-1648b92a2129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:43.509298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1338b0f8-f2e4-4427-b61b-1648b92a2129 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:43.520469 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1187/2371] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:29:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:29:44.393926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:45.138799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d5382-8a06-4a38-94d6-b01e54de1d8a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:45.139194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d5382-8a06-4a38-94d6-b01e54de1d8a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:45.139580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6d5382-8a06-4a38-94d6-b01e54de1d8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:45.160783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7b6d5382-8a06-4a38-94d6-b01e54de1d8a None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:29:45.162141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7b6d5382-8a06-4a38-94d6-b01e54de1d8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:45.224020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1185/2372] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 19:29:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:29:45.282890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338b0f8-f2e4-4427-b61b-1648b92a2129 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:29:45.284393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1338b0f8-f2e4-4427-b61b-1648b92a2129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:29:45.364260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1338b0f8-f2e4-4427-b61b-1648b92a2129 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:29:45.369472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d667f3ab-77b9-49f8-804b-d8b03065174e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:29:45.370176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667f3ab-77b9-49f8-804b-d8b03065174e service swift] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:29:45.370539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667f3ab-77b9-49f8-804b-d8b03065174e service swift] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:29:45.370842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d667f3ab-77b9-49f8-804b-d8b03065174e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:29:45.380163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667f3ab-77b9-49f8-804b-d8b03065174e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:29:45.381819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d667f3ab-77b9-49f8-804b-d8b03065174e service swift] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:29:45.391444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1188/2373] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 19:29:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 19:29:55.216073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=25 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:29:55.387042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=62 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:00.259436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14194425-b103-4482-801f-2e4c94477d8e None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:00.260016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14194425-b103-4482-801f-2e4c94477d8e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:00.260016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14194425-b103-4482-801f-2e4c94477d8e None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:00.261368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1186/2374] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:30:00.268440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3661e-7965-4733-a11f-6a48c5a71452 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:00.268907 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3661e-7965-4733-a11f-6a48c5a71452 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:00.269021 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c3661e-7965-4733-a11f-6a48c5a71452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:00.287469 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-93c3661e-7965-4733-a11f-6a48c5a71452 None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:00.288696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-93c3661e-7965-4733-a11f-6a48c5a71452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:00.298915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1189/2375] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:30:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:00.310644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14194425-b103-4482-801f-2e4c94477d8e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:00.313800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-14194425-b103-4482-801f-2e4c94477d8e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:00.374700 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14194425-b103-4482-801f-2e4c94477d8e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:00.379167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc83b89-0187-4b5c-adcf-00da7f70cc78 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e46ec9c31094d03b6a413e09bc9f699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:00.379599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc83b89-0187-4b5c-adcf-00da7f70cc78 service glance] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:00.379976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc83b89-0187-4b5c-adcf-00da7f70cc78 service glance] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:00.380345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc83b89-0187-4b5c-adcf-00da7f70cc78 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:00.394083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc83b89-0187-4b5c-adcf-00da7f70cc78 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:00.395729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc83b89-0187-4b5c-adcf-00da7f70cc78 service glance] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:00.407150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1187/2376] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:00.479298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60843df6-f67d-4921-bac5-49a930b9f7c5 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:00.479587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60843df6-f67d-4921-bac5-49a930b9f7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:00.479865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60843df6-f67d-4921-bac5-49a930b9f7c5 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:00.481389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1190/2377] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:30:00.487072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e691e-a41d-4a76-b2f4-e7ba36fc6b93 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:00.487431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e691e-a41d-4a76-b2f4-e7ba36fc6b93 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:00.487772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476e691e-a41d-4a76-b2f4-e7ba36fc6b93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:00.507421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-476e691e-a41d-4a76-b2f4-e7ba36fc6b93 None None] MFA Rules not processed for user `3a1a88089d384964bd9774eda4c5d8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:00.508721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-476e691e-a41d-4a76-b2f4-e7ba36fc6b93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:00.515889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1188/2378] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:30:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:00.528066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60843df6-f67d-4921-bac5-49a930b9f7c5 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:00.540068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60843df6-f67d-4921-bac5-49a930b9f7c5 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:00.543944 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2ef015e-932f-408f-90a2-4eabb7ef2f86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:00.544610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef015e-932f-408f-90a2-4eabb7ef2f86 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:00.544891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef015e-932f-408f-90a2-4eabb7ef2f86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:00.545166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2ef015e-932f-408f-90a2-4eabb7ef2f86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:00.552905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef015e-932f-408f-90a2-4eabb7ef2f86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:00.554311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2ef015e-932f-408f-90a2-4eabb7ef2f86 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:00.562333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1191/2379] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:00.650055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d55a9-4744-43ad-b7fc-662a7987bf00 None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:00.650349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d55a9-4744-43ad-b7fc-662a7987bf00 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:00.650629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5d55a9-4744-43ad-b7fc-662a7987bf00 None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:00.652011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1189/2380] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 19:30:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 19:30:00.658332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d7b45-570c-4df1-a558-c7d642350653 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:00.658619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d7b45-570c-4df1-a558-c7d642350653 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:00.658894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339d7b45-570c-4df1-a558-c7d642350653 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:00.676845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-339d7b45-570c-4df1-a558-c7d642350653 None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 19:30:00.678092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-339d7b45-570c-4df1-a558-c7d642350653 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:30:00.687868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1192/2381] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 19:30:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 19:30:00.698284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d55a9-4744-43ad-b7fc-662a7987bf00 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:30:00.710036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5d55a9-4744-43ad-b7fc-662a7987bf00 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:30:00.713998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f6ca8-9157-4288-b96c-3c29c53625cb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:30:00.714671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f6ca8-9157-4288-b96c-3c29c53625cb service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:30:00.714963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f6ca8-9157-4288-b96c-3c29c53625cb service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:30:00.715245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a07f6ca8-9157-4288-b96c-3c29c53625cb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:30:00.726498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f6ca8-9157-4288-b96c-3c29c53625cb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:30:00.727940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a07f6ca8-9157-4288-b96c-3c29c53625cb service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:30:00.739275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1190/2382] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:30:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:30:10.672328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=12 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:30:10.734644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=35 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:02.212743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d7b45-570c-4df1-a558-c7d642350653 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:02.222153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339d7b45-570c-4df1-a558-c7d642350653 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:02.225782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799e716-7e9e-434b-b993-f69324b8e214 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:02.226422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799e716-7e9e-434b-b993-f69324b8e214 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:02.226706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799e716-7e9e-434b-b993-f69324b8e214 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:02.226983 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b799e716-7e9e-434b-b993-f69324b8e214 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:02.235957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799e716-7e9e-434b-b993-f69324b8e214 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:02.237381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b799e716-7e9e-434b-b993-f69324b8e214 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:02.246927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1193/2383] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:02 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:03.722745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f6ca8-9157-4288-b96c-3c29c53625cb service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:03.735305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a07f6ca8-9157-4288-b96c-3c29c53625cb service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:03.741248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075afe1-1006-4551-81d6-098046d526df service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:03.742116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075afe1-1006-4551-81d6-098046d526df service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:03.742283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075afe1-1006-4551-81d6-098046d526df service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:03.742484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075afe1-1006-4551-81d6-098046d526df service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:03.744300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f075afe1-1006-4551-81d6-098046d526df service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:03.806026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075afe1-1006-4551-81d6-098046d526df service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:03.807545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075afe1-1006-4551-81d6-098046d526df service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:03.818332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1191/2384] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:03 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:04.210993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799e716-7e9e-434b-b993-f69324b8e214 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:04.226068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b799e716-7e9e-434b-b993-f69324b8e214 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:04.232959 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04bea0-88ec-4160-9bbb-770995ac5f04 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:04.234400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04bea0-88ec-4160-9bbb-770995ac5f04 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:04.234855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04bea0-88ec-4160-9bbb-770995ac5f04 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:04.235293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e04bea0-88ec-4160-9bbb-770995ac5f04 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:04.237719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9e04bea0-88ec-4160-9bbb-770995ac5f04 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:04.250931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04bea0-88ec-4160-9bbb-770995ac5f04 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:04.252683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e04bea0-88ec-4160-9bbb-770995ac5f04 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:04.263285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1194/2385] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:13.814334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=28 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:14.258767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=12 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:39.980888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075afe1-1006-4551-81d6-098046d526df service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:40.053466 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075afe1-1006-4551-81d6-098046d526df service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:40.057148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ee6ca-0975-4384-b91f-c1b9733c3112 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:40.057828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ee6ca-0975-4384-b91f-c1b9733c3112 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:40.058115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ee6ca-0975-4384-b91f-c1b9733c3112 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:40.058392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-920ee6ca-0975-4384-b91f-c1b9733c3112 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:40.060141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-920ee6ca-0975-4384-b91f-c1b9733c3112 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:40.095505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ee6ca-0975-4384-b91f-c1b9733c3112 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:40.096933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-920ee6ca-0975-4384-b91f-c1b9733c3112 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:40.105789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1192/2386] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:39 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:42.436513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04bea0-88ec-4160-9bbb-770995ac5f04 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:31:42.499034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e04bea0-88ec-4160-9bbb-770995ac5f04 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:31:42.505301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f0c8-4280-4aa8-aafe-f10a57ce2fb7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:31:42.505987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f0c8-4280-4aa8-aafe-f10a57ce2fb7 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:31:42.506276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f0c8-4280-4aa8-aafe-f10a57ce2fb7 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:31:42.506562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2457f0c8-4280-4aa8-aafe-f10a57ce2fb7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:31:42.508376 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2457f0c8-4280-4aa8-aafe-f10a57ce2fb7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:31:42.534639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457f0c8-4280-4aa8-aafe-f10a57ce2fb7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:31:42.536245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2457f0c8-4280-4aa8-aafe-f10a57ce2fb7 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:31:42.549723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1195/2387] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:31:42 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:31:50.101771 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=45 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:31:52.542422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=32 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:27.147054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ee6ca-0975-4384-b91f-c1b9733c3112 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:27.176367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-920ee6ca-0975-4384-b91f-c1b9733c3112 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:27.181885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308800e-480d-4057-ace2-55b2bd53236c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:27.182527 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308800e-480d-4057-ace2-55b2bd53236c service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:27.182814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308800e-480d-4057-ace2-55b2bd53236c service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:27.183095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f308800e-480d-4057-ace2-55b2bd53236c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:27.184893 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f308800e-480d-4057-ace2-55b2bd53236c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:33:27.196019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308800e-480d-4057-ace2-55b2bd53236c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:27.197707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f308800e-480d-4057-ace2-55b2bd53236c service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:27.207025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1193/2388] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:37.202505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=13 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:43.755952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f0c8-4280-4aa8-aafe-f10a57ce2fb7 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:43.775624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2457f0c8-4280-4aa8-aafe-f10a57ce2fb7 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:43.779416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ff2f0-9c6e-4668-b44d-ccf7afdf5eff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:43.780049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ff2f0-9c6e-4668-b44d-ccf7afdf5eff service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:43.780329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ff2f0-9c6e-4668-b44d-ccf7afdf5eff service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:43.780599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2ff2f0-9c6e-4668-b44d-ccf7afdf5eff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:43.792660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ff2f0-9c6e-4668-b44d-ccf7afdf5eff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:43.794051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d2ff2f0-9c6e-4668-b44d-ccf7afdf5eff service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:43.806176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1196/2389] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:48.577371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308800e-480d-4057-ace2-55b2bd53236c service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:48.587123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f308800e-480d-4057-ace2-55b2bd53236c service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:48.590690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212ca27-2707-4d33-bbc4-92da47011cdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:48.591332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212ca27-2707-4d33-bbc4-92da47011cdf service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:48.591618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212ca27-2707-4d33-bbc4-92da47011cdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:48.591899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7212ca27-2707-4d33-bbc4-92da47011cdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:48.602919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212ca27-2707-4d33-bbc4-92da47011cdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:48.604347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7212ca27-2707-4d33-bbc4-92da47011cdf service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:48.615999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1194/2390] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:48 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:53.801163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=10 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:33:54.438531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ff2f0-9c6e-4668-b44d-ccf7afdf5eff service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:33:54.449781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d2ff2f0-9c6e-4668-b44d-ccf7afdf5eff service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:33:54.455562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f68e9-3e22-4fa5-ac46-b7c89fb9ed46 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:33:54.456191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f68e9-3e22-4fa5-ac46-b7c89fb9ed46 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:33:54.456475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f68e9-3e22-4fa5-ac46-b7c89fb9ed46 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:33:54.456753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-950f68e9-3e22-4fa5-ac46-b7c89fb9ed46 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:33:54.465009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f68e9-3e22-4fa5-ac46-b7c89fb9ed46 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:33:54.466601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-950f68e9-3e22-4fa5-ac46-b7c89fb9ed46 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:33:54.494331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1197/2391] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:33:54 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:33:58.611278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:04.491791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=13 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:04.663753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212ca27-2707-4d33-bbc4-92da47011cdf service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:04.675739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7212ca27-2707-4d33-bbc4-92da47011cdf service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:04.681209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0dc5-2c3b-4e2b-99a4-da25ebc3533c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:04.681883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0dc5-2c3b-4e2b-99a4-da25ebc3533c service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:04.682165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0dc5-2c3b-4e2b-99a4-da25ebc3533c service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:04.682446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dde0dc5-2c3b-4e2b-99a4-da25ebc3533c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:04.692060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde0dc5-2c3b-4e2b-99a4-da25ebc3533c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:04.693702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dde0dc5-2c3b-4e2b-99a4-da25ebc3533c service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:04.703139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1195/2392] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:04 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:34:14.698840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:20.051769 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f68e9-3e22-4fa5-ac46-b7c89fb9ed46 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:20.062732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-950f68e9-3e22-4fa5-ac46-b7c89fb9ed46 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:20.067646 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4494-f35a-4a47-b12b-987b6ed33cf6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:20.068403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4494-f35a-4a47-b12b-987b6ed33cf6 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:20.068743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4494-f35a-4a47-b12b-987b6ed33cf6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:20.069076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceda4494-f35a-4a47-b12b-987b6ed33cf6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:20.077774 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda4494-f35a-4a47-b12b-987b6ed33cf6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:20.079200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceda4494-f35a-4a47-b12b-987b6ed33cf6 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:20.088932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1198/2393] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:34:25.894395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0dc5-2c3b-4e2b-99a4-da25ebc3533c service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:25.906897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dde0dc5-2c3b-4e2b-99a4-da25ebc3533c service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:25.913018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668912e-dc17-448c-9908-f712f8209db2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:25.913715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668912e-dc17-448c-9908-f712f8209db2 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:25.914125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668912e-dc17-448c-9908-f712f8209db2 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:25.914401 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b668912e-dc17-448c-9908-f712f8209db2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:25.924775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668912e-dc17-448c-9908-f712f8209db2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:25.926713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b668912e-dc17-448c-9908-f712f8209db2 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:25.938762 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1196/2394] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:34:30.084222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:35.934183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:34:55.715524 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4494-f35a-4a47-b12b-987b6ed33cf6 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:34:55.736860 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceda4494-f35a-4a47-b12b-987b6ed33cf6 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:34:55.740825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c6256-0383-4b5e-9bc9-91bb50bfc183 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:34:55.741486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c6256-0383-4b5e-9bc9-91bb50bfc183 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:34:55.741860 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c6256-0383-4b5e-9bc9-91bb50bfc183 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:34:55.742145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0c6256-0383-4b5e-9bc9-91bb50bfc183 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:34:55.751862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c6256-0383-4b5e-9bc9-91bb50bfc183 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:34:55.753535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0c6256-0383-4b5e-9bc9-91bb50bfc183 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:34:55.782747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1199/2395] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:34:55 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:35:05.779895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=17 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:36:49.417944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668912e-dc17-448c-9908-f712f8209db2 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:49.419325 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b668912e-dc17-448c-9908-f712f8209db2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:36:49.434171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b668912e-dc17-448c-9908-f712f8209db2 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:49.439112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963e9ae-edca-440c-b036-b3c62519f0ce service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:49.439752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963e9ae-edca-440c-b036-b3c62519f0ce service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:49.440038 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963e9ae-edca-440c-b036-b3c62519f0ce service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:49.440340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e963e9ae-edca-440c-b036-b3c62519f0ce service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:49.448773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963e9ae-edca-440c-b036-b3c62519f0ce service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:49.450390 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e963e9ae-edca-440c-b036-b3c62519f0ce service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:49.460405 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1197/2396] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:56.385054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c6256-0383-4b5e-9bc9-91bb50bfc183 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:56.393747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0c6256-0383-4b5e-9bc9-91bb50bfc183 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:56.397395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd67a2-435d-44b2-87a6-8ae3d566e3f2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:56.398063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd67a2-435d-44b2-87a6-8ae3d566e3f2 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:56.398346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd67a2-435d-44b2-87a6-8ae3d566e3f2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:56.398625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bfd67a2-435d-44b2-87a6-8ae3d566e3f2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:56.407213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd67a2-435d-44b2-87a6-8ae3d566e3f2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:56.408624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bfd67a2-435d-44b2-87a6-8ae3d566e3f2 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:56.437778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1200/2397] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:36:56.483244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963e9ae-edca-440c-b036-b3c62519f0ce service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:36:56.492815 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e963e9ae-edca-440c-b036-b3c62519f0ce service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:36:56.496357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f13c6-27ad-432d-8dc9-13b02b89ca0a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:36:56.496992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f13c6-27ad-432d-8dc9-13b02b89ca0a service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:36:56.497273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f13c6-27ad-432d-8dc9-13b02b89ca0a service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:36:56.497584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8f13c6-27ad-432d-8dc9-13b02b89ca0a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:36:56.506530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f13c6-27ad-432d-8dc9-13b02b89ca0a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:36:56.507966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8f13c6-27ad-432d-8dc9-13b02b89ca0a service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:36:56.535255 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1198/2398] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:36:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:37:06.435183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=13 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:37:06.532565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=19 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:27.744691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd67a2-435d-44b2-87a6-8ae3d566e3f2 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:27.758202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bfd67a2-435d-44b2-87a6-8ae3d566e3f2 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:27.763767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c32f4e-0f06-4961-b562-af3c57c34b95 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:27.764407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c32f4e-0f06-4961-b562-af3c57c34b95 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:27.764694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c32f4e-0f06-4961-b562-af3c57c34b95 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:27.765000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18c32f4e-0f06-4961-b562-af3c57c34b95 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:27.773820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c32f4e-0f06-4961-b562-af3c57c34b95 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:27.775401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18c32f4e-0f06-4961-b562-af3c57c34b95 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:27.802487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1201/2399] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:27 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:37.800446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=13 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:44.176642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f13c6-27ad-432d-8dc9-13b02b89ca0a service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:44.178313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-da8f13c6-27ad-432d-8dc9-13b02b89ca0a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:44.202887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8f13c6-27ad-432d-8dc9-13b02b89ca0a service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:44.206891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2968365-9c15-4c07-be81-ff24c14950e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:44.207584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968365-9c15-4c07-be81-ff24c14950e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:44.207869 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968365-9c15-4c07-be81-ff24c14950e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:44.208178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2968365-9c15-4c07-be81-ff24c14950e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:44.210342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a2968365-9c15-4c07-be81-ff24c14950e2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:44.236177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2968365-9c15-4c07-be81-ff24c14950e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:44.238315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2968365-9c15-4c07-be81-ff24c14950e2 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:44.249032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1199/2400] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:38:54.244223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=16 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:38:56.686017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c32f4e-0f06-4961-b562-af3c57c34b95 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:38:56.687587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-18c32f4e-0f06-4961-b562-af3c57c34b95 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:56.714283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18c32f4e-0f06-4961-b562-af3c57c34b95 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:38:56.717825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270afca6-ed10-4b65-820c-2aaa2dea1340 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:38:56.718465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270afca6-ed10-4b65-820c-2aaa2dea1340 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:38:56.718745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270afca6-ed10-4b65-820c-2aaa2dea1340 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:38:56.719270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270afca6-ed10-4b65-820c-2aaa2dea1340 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:38:56.721087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-270afca6-ed10-4b65-820c-2aaa2dea1340 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:38:56.915513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270afca6-ed10-4b65-820c-2aaa2dea1340 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:38:56.916931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270afca6-ed10-4b65-820c-2aaa2dea1340 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:38:56.926028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1202/2401] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:38:56 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:39:02.989168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2968365-9c15-4c07-be81-ff24c14950e2 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:39:03.042567 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2968365-9c15-4c07-be81-ff24c14950e2 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:39:03.046386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6abd4-949d-4fee-b372-fc4f5ad4d669 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:39:03.047033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6abd4-949d-4fee-b372-fc4f5ad4d669 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:39:03.047317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6abd4-949d-4fee-b372-fc4f5ad4d669 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:39:03.047595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63e6abd4-949d-4fee-b372-fc4f5ad4d669 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:39:03.049677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-63e6abd4-949d-4fee-b372-fc4f5ad4d669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:39:03.130213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6abd4-949d-4fee-b372-fc4f5ad4d669 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:39:03.131653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63e6abd4-949d-4fee-b372-fc4f5ad4d669 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:39:03.150060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1200/2402] 10.222.0.69 () {60 vars in 1396 bytes} [Wed Jul 20 19:39:02 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3240 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:39:06.921737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=38 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:39:13.147464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=61 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:39:46.114938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270afca6-ed10-4b65-820c-2aaa2dea1340 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:39:46.127284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270afca6-ed10-4b65-820c-2aaa2dea1340 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:39:46.131132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b665e0-efe1-47b6-9526-506f76bdcd1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:39:46.131807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b665e0-efe1-47b6-9526-506f76bdcd1d service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:39:46.132091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b665e0-efe1-47b6-9526-506f76bdcd1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:39:46.132370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b665e0-efe1-47b6-9526-506f76bdcd1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:39:46.134391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-88b665e0-efe1-47b6-9526-506f76bdcd1d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:39:46.144701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b665e0-efe1-47b6-9526-506f76bdcd1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:39:46.146206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b665e0-efe1-47b6-9526-506f76bdcd1d service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:39:46.154516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1203/2403] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:39:46 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:39:56.150786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:40:06.685559 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6abd4-949d-4fee-b372-fc4f5ad4d669 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:40:06.687428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-63e6abd4-949d-4fee-b372-fc4f5ad4d669 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:40:06.770103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63e6abd4-949d-4fee-b372-fc4f5ad4d669 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:40:06.775624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9211f-f8e9-4a06-a61b-9c746c514002 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:40:06.776287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9211f-f8e9-4a06-a61b-9c746c514002 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:40:06.776572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9211f-f8e9-4a06-a61b-9c746c514002 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:40:06.776851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef9211f-f8e9-4a06-a61b-9c746c514002 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:40:06.778817 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0ef9211f-f8e9-4a06-a61b-9c746c514002 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:40:06.792421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9211f-f8e9-4a06-a61b-9c746c514002 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:40:06.794080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ef9211f-f8e9-4a06-a61b-9c746c514002 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:40:06.805221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1201/2404] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:40:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:40:16.801196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=34 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:40:57.004950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b665e0-efe1-47b6-9526-506f76bdcd1d service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:40:57.014508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b665e0-efe1-47b6-9526-506f76bdcd1d service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:40:57.019005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fc3c9-6a50-47ea-9cbc-17cc7c43e485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:40:57.019525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fc3c9-6a50-47ea-9cbc-17cc7c43e485 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:40:57.019807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fc3c9-6a50-47ea-9cbc-17cc7c43e485 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:40:57.020079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3fc3c9-6a50-47ea-9cbc-17cc7c43e485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:40:57.022079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9d3fc3c9-6a50-47ea-9cbc-17cc7c43e485 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:40:57.098113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fc3c9-6a50-47ea-9cbc-17cc7c43e485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:40:57.099694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3fc3c9-6a50-47ea-9cbc-17cc7c43e485 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:40:57.109937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1204/2405] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:40:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:41:07.104687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=33 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:41:55.377859 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9211f-f8e9-4a06-a61b-9c746c514002 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:41:55.379678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0ef9211f-f8e9-4a06-a61b-9c746c514002 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:55.519128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef9211f-f8e9-4a06-a61b-9c746c514002 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:41:55.524680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb72f9-3b7e-4751-a73a-5dd033a5367d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:41:55.525330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb72f9-3b7e-4751-a73a-5dd033a5367d service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:41:55.525654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb72f9-3b7e-4751-a73a-5dd033a5367d service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:41:55.525937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51cb72f9-3b7e-4751-a73a-5dd033a5367d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:41:55.527839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-51cb72f9-3b7e-4751-a73a-5dd033a5367d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:41:55.539347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb72f9-3b7e-4751-a73a-5dd033a5367d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:41:55.540932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51cb72f9-3b7e-4751-a73a-5dd033a5367d service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:41:55.550799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1202/2406] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:41:55 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:42:05.546560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=63 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:42:57.324266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fc3c9-6a50-47ea-9cbc-17cc7c43e485 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:57.393095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3fc3c9-6a50-47ea-9cbc-17cc7c43e485 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:57.397556 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216ea34-27f6-425d-85a6-d4e5c6ee8a40 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:57.398205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216ea34-27f6-425d-85a6-d4e5c6ee8a40 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:57.398487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216ea34-27f6-425d-85a6-d4e5c6ee8a40 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:57.398764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216ea34-27f6-425d-85a6-d4e5c6ee8a40 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:57.400688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6216ea34-27f6-425d-85a6-d4e5c6ee8a40 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:57.477636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216ea34-27f6-425d-85a6-d4e5c6ee8a40 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:57.479061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216ea34-27f6-425d-85a6-d4e5c6ee8a40 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:57.488301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1205/2407] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:42:57 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:42:57.533220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb72f9-3b7e-4751-a73a-5dd033a5367d service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:42:57.543111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51cb72f9-3b7e-4751-a73a-5dd033a5367d service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:42:57.546891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace1123-b30a-49d6-95aa-39428d0d8951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:42:57.547607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace1123-b30a-49d6-95aa-39428d0d8951 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:42:57.547962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace1123-b30a-49d6-95aa-39428d0d8951 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:42:57.548334 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cace1123-b30a-49d6-95aa-39428d0d8951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:42:57.550360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cace1123-b30a-49d6-95aa-39428d0d8951 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:42:57.624992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace1123-b30a-49d6-95aa-39428d0d8951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:42:57.626498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cace1123-b30a-49d6-95aa-39428d0d8951 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:42:57.637602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1203/2408] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:42:57 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:43:07.483767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=58 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:43:07.632503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=33 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:43:26.911063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216ea34-27f6-425d-85a6-d4e5c6ee8a40 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:43:26.925874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216ea34-27f6-425d-85a6-d4e5c6ee8a40 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:43:26.932591 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c144575-2cc6-4939-b16b-70c0f444ad7c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:43:26.933584 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c144575-2cc6-4939-b16b-70c0f444ad7c service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:43:26.933871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c144575-2cc6-4939-b16b-70c0f444ad7c service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:43:26.934147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c144575-2cc6-4939-b16b-70c0f444ad7c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:43:26.994416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c144575-2cc6-4939-b16b-70c0f444ad7c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:43:26.995935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c144575-2cc6-4939-b16b-70c0f444ad7c service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:43:27.005608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1206/2409] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:43:26 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:43:37.001260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=28 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:43:43.164133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace1123-b30a-49d6-95aa-39428d0d8951 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:43:43.173767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cace1123-b30a-49d6-95aa-39428d0d8951 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:43:43.177422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102734e9-2c1b-4539-b1e9-de6e60471ab1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:43:43.178143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102734e9-2c1b-4539-b1e9-de6e60471ab1 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:43:43.178425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102734e9-2c1b-4539-b1e9-de6e60471ab1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:43:43.178705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102734e9-2c1b-4539-b1e9-de6e60471ab1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:43:43.200689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102734e9-2c1b-4539-b1e9-de6e60471ab1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:43:43.202346 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-102734e9-2c1b-4539-b1e9-de6e60471ab1 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:43:43.225225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1204/2410] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:43:43 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 19:43:53.221436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=13 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:44:03.260148 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c144575-2cc6-4939-b16b-70c0f444ad7c service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:44:03.290398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c144575-2cc6-4939-b16b-70c0f444ad7c service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:44:03.295914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0243a7-a97a-49e0-afe3-83d69eeceb3a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:44:03.296560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0243a7-a97a-49e0-afe3-83d69eeceb3a service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:44:03.296846 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0243a7-a97a-49e0-afe3-83d69eeceb3a service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:44:03.297124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0243a7-a97a-49e0-afe3-83d69eeceb3a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:44:03.305396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0243a7-a97a-49e0-afe3-83d69eeceb3a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:44:03.307011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0243a7-a97a-49e0-afe3-83d69eeceb3a service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:44:03.316743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1207/2411] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:44:03 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:44:13.311989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=13 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:44:54.087041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102734e9-2c1b-4539-b1e9-de6e60471ab1 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:44:54.105735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102734e9-2c1b-4539-b1e9-de6e60471ab1 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:44:54.109490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96451036-133e-44d1-9397-f8f91b2beeda service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:44:54.110168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96451036-133e-44d1-9397-f8f91b2beeda service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:44:54.110449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96451036-133e-44d1-9397-f8f91b2beeda service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:44:54.110726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96451036-133e-44d1-9397-f8f91b2beeda service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:44:54.118798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96451036-133e-44d1-9397-f8f91b2beeda service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:44:54.120215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96451036-133e-44d1-9397-f8f91b2beeda service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:44:54.128085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1205/2412] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:44:54 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:44:57.843804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0243a7-a97a-49e0-afe3-83d69eeceb3a service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:44:57.865909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0243a7-a97a-49e0-afe3-83d69eeceb3a service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:44:57.869669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce7c6e-2ca5-4408-a2f5-2e6e25c94625 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:44:57.870319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce7c6e-2ca5-4408-a2f5-2e6e25c94625 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:44:57.870607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce7c6e-2ca5-4408-a2f5-2e6e25c94625 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:44:57.870885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ce7c6e-2ca5-4408-a2f5-2e6e25c94625 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:44:57.880300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce7c6e-2ca5-4408-a2f5-2e6e25c94625 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:44:57.881755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34ce7c6e-2ca5-4408-a2f5-2e6e25c94625 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:44:57.916521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1208/2413] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:44:57 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:45:04.124503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=10 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:45:07.914165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=17 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:46:59.365650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96451036-133e-44d1-9397-f8f91b2beeda service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:46:59.367375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-96451036-133e-44d1-9397-f8f91b2beeda service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:46:59.383788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96451036-133e-44d1-9397-f8f91b2beeda service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:46:59.388693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bfc6e-8c4a-46b0-b4ee-eb5fd6770ef9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:46:59.389360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bfc6e-8c4a-46b0-b4ee-eb5fd6770ef9 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:46:59.389788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bfc6e-8c4a-46b0-b4ee-eb5fd6770ef9 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:46:59.390161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f02bfc6e-8c4a-46b0-b4ee-eb5fd6770ef9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:46:59.399240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bfc6e-8c4a-46b0-b4ee-eb5fd6770ef9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:46:59.400822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f02bfc6e-8c4a-46b0-b4ee-eb5fd6770ef9 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:46:59.410998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1206/2414] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:46:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:47:09.406757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:47:10.585696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce7c6e-2ca5-4408-a2f5-2e6e25c94625 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:47:10.587113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-34ce7c6e-2ca5-4408-a2f5-2e6e25c94625 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:47:10.599801 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ce7c6e-2ca5-4408-a2f5-2e6e25c94625 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:47:10.603493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0eb8cc-9774-421f-83f9-7ca535c7ba51 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:47:10.604150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0eb8cc-9774-421f-83f9-7ca535c7ba51 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:47:10.604444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0eb8cc-9774-421f-83f9-7ca535c7ba51 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:47:10.604721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b0eb8cc-9774-421f-83f9-7ca535c7ba51 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:47:10.614251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0eb8cc-9774-421f-83f9-7ca535c7ba51 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:47:10.615786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b0eb8cc-9774-421f-83f9-7ca535c7ba51 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:47:10.647601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1209/2415] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:47:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:47:20.645738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=13 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:48:58.100117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bfc6e-8c4a-46b0-b4ee-eb5fd6770ef9 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:48:58.101842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f02bfc6e-8c4a-46b0-b4ee-eb5fd6770ef9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:48:58.130332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f02bfc6e-8c4a-46b0-b4ee-eb5fd6770ef9 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:48:58.133275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080ed6a-fa65-45e0-b4c9-69678b67bde6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:48:58.133962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080ed6a-fa65-45e0-b4c9-69678b67bde6 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:48:58.134268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080ed6a-fa65-45e0-b4c9-69678b67bde6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:48:58.134547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4080ed6a-fa65-45e0-b4c9-69678b67bde6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:48:58.136444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4080ed6a-fa65-45e0-b4c9-69678b67bde6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:48:58.171705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080ed6a-fa65-45e0-b4c9-69678b67bde6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:48:58.173321 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4080ed6a-fa65-45e0-b4c9-69678b67bde6 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:48:58.197458 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1207/2416] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:48:58 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:48:58.286128 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0eb8cc-9774-421f-83f9-7ca535c7ba51 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:48:58.294883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b0eb8cc-9774-421f-83f9-7ca535c7ba51 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:48:58.299541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb201f6-5335-4080-9ac8-5fdd7bd80614 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:48:58.300197 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb201f6-5335-4080-9ac8-5fdd7bd80614 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:48:58.300488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb201f6-5335-4080-9ac8-5fdd7bd80614 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:48:58.300764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb201f6-5335-4080-9ac8-5fdd7bd80614 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:48:58.303126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ebb201f6-5335-4080-9ac8-5fdd7bd80614 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:48:58.332872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb201f6-5335-4080-9ac8-5fdd7bd80614 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:48:58.334568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb201f6-5335-4080-9ac8-5fdd7bd80614 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:48:58.348121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1210/2417] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:48:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:48:58.361845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080ed6a-fa65-45e0-b4c9-69678b67bde6 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:48:58.371139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4080ed6a-fa65-45e0-b4c9-69678b67bde6 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:48:58.376080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828df281-95fd-47c8-8c6c-8eed0fb49fa7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:48:58.376876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828df281-95fd-47c8-8c6c-8eed0fb49fa7 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:48:58.377188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828df281-95fd-47c8-8c6c-8eed0fb49fa7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:48:58.377475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828df281-95fd-47c8-8c6c-8eed0fb49fa7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:48:58.387637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828df281-95fd-47c8-8c6c-8eed0fb49fa7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:48:58.389089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-828df281-95fd-47c8-8c6c-8eed0fb49fa7 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:48:58.422747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1208/2418] 10.222.0.69 () {60 vars in 1396 bytes} [Wed Jul 20 19:48:58 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 67 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:48:59.747712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb201f6-5335-4080-9ac8-5fdd7bd80614 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:48:59.759559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb201f6-5335-4080-9ac8-5fdd7bd80614 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:48:59.765645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ace966-add6-4270-bf45-4f521c52fad3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:48:59.766317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ace966-add6-4270-bf45-4f521c52fad3 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:48:59.766598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ace966-add6-4270-bf45-4f521c52fad3 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:48:59.766871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ace966-add6-4270-bf45-4f521c52fad3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:48:59.774987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ace966-add6-4270-bf45-4f521c52fad3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:48:59.776556 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ace966-add6-4270-bf45-4f521c52fad3 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:48:59.803621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1211/2419] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:48:59 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:49:08.418766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=37 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:49:09.800912 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=24 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:49:11.060214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828df281-95fd-47c8-8c6c-8eed0fb49fa7 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:49:11.071455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828df281-95fd-47c8-8c6c-8eed0fb49fa7 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:49:11.077021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab32699-b564-4d49-bc50-7c645f0043f6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:49:11.077784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab32699-b564-4d49-bc50-7c645f0043f6 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:49:11.078095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab32699-b564-4d49-bc50-7c645f0043f6 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:49:11.078375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab32699-b564-4d49-bc50-7c645f0043f6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:49:11.080286 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-dab32699-b564-4d49-bc50-7c645f0043f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:49:11.159738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab32699-b564-4d49-bc50-7c645f0043f6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:49:11.161337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab32699-b564-4d49-bc50-7c645f0043f6 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:49:11.179713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1209/2420] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:49:11 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:49:21.177029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=41 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:49:58.094605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ace966-add6-4270-bf45-4f521c52fad3 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:49:58.096316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c1ace966-add6-4270-bf45-4f521c52fad3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:49:58.165545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ace966-add6-4270-bf45-4f521c52fad3 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:49:58.169205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94013504-c7ee-40ed-a8fb-c6fd9aec8a86 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:49:58.170395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94013504-c7ee-40ed-a8fb-c6fd9aec8a86 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:49:58.170765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94013504-c7ee-40ed-a8fb-c6fd9aec8a86 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:49:58.171047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94013504-c7ee-40ed-a8fb-c6fd9aec8a86 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:49:58.172934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-94013504-c7ee-40ed-a8fb-c6fd9aec8a86 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:49:58.182843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94013504-c7ee-40ed-a8fb-c6fd9aec8a86 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:49:58.184444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94013504-c7ee-40ed-a8fb-c6fd9aec8a86 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:49:58.193180 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1212/2421] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:49:58 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:50:08.189671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=31 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:51:11.764480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab32699-b564-4d49-bc50-7c645f0043f6 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:51:11.766181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-dab32699-b564-4d49-bc50-7c645f0043f6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:51:11.837380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab32699-b564-4d49-bc50-7c645f0043f6 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:51:11.842910 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c75fa8-a37e-4322-beb1-9389da3c5de9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:51:11.843561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c75fa8-a37e-4322-beb1-9389da3c5de9 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:51:11.843846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c75fa8-a37e-4322-beb1-9389da3c5de9 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:51:11.844123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c75fa8-a37e-4322-beb1-9389da3c5de9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:51:11.846029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-70c75fa8-a37e-4322-beb1-9389da3c5de9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:51:11.858285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c75fa8-a37e-4322-beb1-9389da3c5de9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:51:11.859868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c75fa8-a37e-4322-beb1-9389da3c5de9 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:51:11.871330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1210/2422] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:51:11 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:51:11.909807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94013504-c7ee-40ed-a8fb-c6fd9aec8a86 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:51:11.923025 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94013504-c7ee-40ed-a8fb-c6fd9aec8a86 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:51:11.929120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029755b7-ab01-4172-a2ec-13fd22a23bc6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:51:11.929793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029755b7-ab01-4172-a2ec-13fd22a23bc6 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:51:11.930082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029755b7-ab01-4172-a2ec-13fd22a23bc6 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:51:11.930359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029755b7-ab01-4172-a2ec-13fd22a23bc6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:51:11.932223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-029755b7-ab01-4172-a2ec-13fd22a23bc6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:51:12.002472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029755b7-ab01-4172-a2ec-13fd22a23bc6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:51:12.004056 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-029755b7-ab01-4172-a2ec-13fd22a23bc6 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:51:12.013652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1213/2423] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:51:11 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:51:21.866877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=34 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:51:22.009446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=33 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:52:03.335598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c75fa8-a37e-4322-beb1-9389da3c5de9 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:52:03.337766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-70c75fa8-a37e-4322-beb1-9389da3c5de9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:52:03.465156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c75fa8-a37e-4322-beb1-9389da3c5de9 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:52:03.470361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec25f5-5e0b-4c8a-b6db-8cd529205a6a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:52:03.471012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec25f5-5e0b-4c8a-b6db-8cd529205a6a service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:52:03.471558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec25f5-5e0b-4c8a-b6db-8cd529205a6a service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:52:03.471838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec25f5-5e0b-4c8a-b6db-8cd529205a6a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:52:03.473781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-deec25f5-5e0b-4c8a-b6db-8cd529205a6a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:52:03.485086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec25f5-5e0b-4c8a-b6db-8cd529205a6a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:52:03.486711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec25f5-5e0b-4c8a-b6db-8cd529205a6a service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:52:03.497007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1211/2424] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:52:03 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:52:13.492230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=58 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:53:11.820655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029755b7-ab01-4172-a2ec-13fd22a23bc6 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:53:11.862479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029755b7-ab01-4172-a2ec-13fd22a23bc6 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:53:11.867953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f051ef-eb44-4e63-a8a3-15b2f9c3a9ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:53:11.868590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f051ef-eb44-4e63-a8a3-15b2f9c3a9ef service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:53:11.868871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f051ef-eb44-4e63-a8a3-15b2f9c3a9ef service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:53:11.869146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f051ef-eb44-4e63-a8a3-15b2f9c3a9ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:53:11.871046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-04f051ef-eb44-4e63-a8a3-15b2f9c3a9ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:53:11.941170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f051ef-eb44-4e63-a8a3-15b2f9c3a9ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:53:11.942701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f051ef-eb44-4e63-a8a3-15b2f9c3a9ef service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:53:11.952381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1214/2425] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:53:11 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:53:21.947520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=47 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:54:59.390197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec25f5-5e0b-4c8a-b6db-8cd529205a6a service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:54:59.462808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec25f5-5e0b-4c8a-b6db-8cd529205a6a service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:54:59.466503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb010fe-58e7-4e0b-82ce-fdcc57ed0cbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:54:59.467639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb010fe-58e7-4e0b-82ce-fdcc57ed0cbe service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:54:59.467920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb010fe-58e7-4e0b-82ce-fdcc57ed0cbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:54:59.468196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb010fe-58e7-4e0b-82ce-fdcc57ed0cbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:54:59.485570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb010fe-58e7-4e0b-82ce-fdcc57ed0cbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:54:59.487015 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb010fe-58e7-4e0b-82ce-fdcc57ed0cbe service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:54:59.507942 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1212/2426] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:54:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:54:59.515573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f051ef-eb44-4e63-a8a3-15b2f9c3a9ef service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:54:59.524708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f051ef-eb44-4e63-a8a3-15b2f9c3a9ef service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:54:59.528278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b4ff2-ec6b-4aa9-9160-f1409a29c567 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:54:59.528962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b4ff2-ec6b-4aa9-9160-f1409a29c567 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:54:59.529777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b4ff2-ec6b-4aa9-9160-f1409a29c567 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:54:59.530150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08b4ff2-ec6b-4aa9-9160-f1409a29c567 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:54:59.532164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a08b4ff2-ec6b-4aa9-9160-f1409a29c567 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:54:59.602674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b4ff2-ec6b-4aa9-9160-f1409a29c567 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:54:59.604034 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08b4ff2-ec6b-4aa9-9160-f1409a29c567 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:54:59.613291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1215/2427] 10.222.0.69 () {60 vars in 1396 bytes} [Wed Jul 20 19:54:59 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:54:59.661885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb010fe-58e7-4e0b-82ce-fdcc57ed0cbe service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:54:59.670445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb010fe-58e7-4e0b-82ce-fdcc57ed0cbe service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:54:59.674045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597ccc1-7e53-4c9a-8dbd-fc8e7fe9dbdd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:54:59.674693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597ccc1-7e53-4c9a-8dbd-fc8e7fe9dbdd service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:54:59.674977 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597ccc1-7e53-4c9a-8dbd-fc8e7fe9dbdd service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:54:59.675252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7597ccc1-7e53-4c9a-8dbd-fc8e7fe9dbdd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:54:59.732152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597ccc1-7e53-4c9a-8dbd-fc8e7fe9dbdd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:54:59.733462 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7597ccc1-7e53-4c9a-8dbd-fc8e7fe9dbdd service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:54:59.742353 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1213/2428] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:54:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:54:59.833496 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b4ff2-ec6b-4aa9-9160-f1409a29c567 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:54:59.842828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08b4ff2-ec6b-4aa9-9160-f1409a29c567 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:54:59.846864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abfb02-7efb-4e11-8fb5-c7c33a2a811a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:54:59.847508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfb02-7efb-4e11-8fb5-c7c33a2a811a service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:54:59.847795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfb02-7efb-4e11-8fb5-c7c33a2a811a service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:54:59.848073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4abfb02-7efb-4e11-8fb5-c7c33a2a811a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:54:59.856763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abfb02-7efb-4e11-8fb5-c7c33a2a811a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:54:59.858479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4abfb02-7efb-4e11-8fb5-c7c33a2a811a service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:54:59.867535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1216/2429] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:54:59 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:55:06.070031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597ccc1-7e53-4c9a-8dbd-fc8e7fe9dbdd service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:55:06.088708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7597ccc1-7e53-4c9a-8dbd-fc8e7fe9dbdd service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:55:06.092892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc26d56-45ee-4ab2-a747-0cf58c155d8a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:55:06.093535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc26d56-45ee-4ab2-a747-0cf58c155d8a service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:55:06.093825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc26d56-45ee-4ab2-a747-0cf58c155d8a service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:55:06.094095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc26d56-45ee-4ab2-a747-0cf58c155d8a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:55:06.101460 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc26d56-45ee-4ab2-a747-0cf58c155d8a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:55:06.102911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdc26d56-45ee-4ab2-a747-0cf58c155d8a service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:55:06.111331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1214/2430] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:55:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:55:09.863443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=39 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:55:11.903083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abfb02-7efb-4e11-8fb5-c7c33a2a811a service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:55:11.916903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4abfb02-7efb-4e11-8fb5-c7c33a2a811a service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:55:11.922520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa27ed5-093f-4803-999e-b42fef338f2f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:55:11.923363 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa27ed5-093f-4803-999e-b42fef338f2f service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:55:11.923629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa27ed5-093f-4803-999e-b42fef338f2f service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:55:11.923908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa27ed5-093f-4803-999e-b42fef338f2f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:55:11.925822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-caa27ed5-093f-4803-999e-b42fef338f2f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:55:11.996844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa27ed5-093f-4803-999e-b42fef338f2f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:55:11.998457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa27ed5-093f-4803-999e-b42fef338f2f service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:55:12.008056 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1217/2431] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:55:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:55:16.107214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=80 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:55:22.003836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=33 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:57:09.391391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc26d56-45ee-4ab2-a747-0cf58c155d8a service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:57:09.392949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bdc26d56-45ee-4ab2-a747-0cf58c155d8a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:57:09.408017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdc26d56-45ee-4ab2-a747-0cf58c155d8a service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:57:09.413683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b4e2a-e023-4077-84c3-565f84c61225 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:57:09.414289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b4e2a-e023-4077-84c3-565f84c61225 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:57:09.414571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b4e2a-e023-4077-84c3-565f84c61225 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:57:09.414844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84b4e2a-e023-4077-84c3-565f84c61225 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:57:09.423555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b4e2a-e023-4077-84c3-565f84c61225 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:57:09.425119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84b4e2a-e023-4077-84c3-565f84c61225 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:57:09.435729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1215/2432] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:57:09 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:57:12.001574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa27ed5-093f-4803-999e-b42fef338f2f service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:57:12.019215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa27ed5-093f-4803-999e-b42fef338f2f service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:57:12.029992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86fff1-4ccf-4e71-9ae8-c441353fa597 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:57:12.031056 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86fff1-4ccf-4e71-9ae8-c441353fa597 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:57:12.031737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86fff1-4ccf-4e71-9ae8-c441353fa597 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:57:12.032134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86fff1-4ccf-4e71-9ae8-c441353fa597 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:57:12.048067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86fff1-4ccf-4e71-9ae8-c441353fa597 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:57:12.050937 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86fff1-4ccf-4e71-9ae8-c441353fa597 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:57:12.070663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1218/2433] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:57:12 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:57:13.168038 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b4e2a-e023-4077-84c3-565f84c61225 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:57:13.181017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84b4e2a-e023-4077-84c3-565f84c61225 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:57:13.186568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd965-3ad4-49ac-b677-002b4020bcea service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:57:13.187215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd965-3ad4-49ac-b677-002b4020bcea service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:57:13.187502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd965-3ad4-49ac-b677-002b4020bcea service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:57:13.187783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ccd965-3ad4-49ac-b677-002b4020bcea service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:57:13.196678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccd965-3ad4-49ac-b677-002b4020bcea service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:57:13.198316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ccd965-3ad4-49ac-b677-002b4020bcea service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:57:13.227089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1216/2434] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:57:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:57:22.062072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:57:23.223829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=19 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:59:00.400334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86fff1-4ccf-4e71-9ae8-c441353fa597 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:00.411304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86fff1-4ccf-4e71-9ae8-c441353fa597 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:00.417451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9829d4d-0783-492a-8e80-21a94a7dcbcd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:00.418107 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829d4d-0783-492a-8e80-21a94a7dcbcd service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:00.418386 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829d4d-0783-492a-8e80-21a94a7dcbcd service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:00.418660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9829d4d-0783-492a-8e80-21a94a7dcbcd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:00.426498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9829d4d-0783-492a-8e80-21a94a7dcbcd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:00.428261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9829d4d-0783-492a-8e80-21a94a7dcbcd service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:00.454784 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1219/2435] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:59:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:59:02.158991 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd965-3ad4-49ac-b677-002b4020bcea service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:02.173580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ccd965-3ad4-49ac-b677-002b4020bcea service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:02.180095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b07d1-e717-40c1-a8a9-bfa81c288b5f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:02.180995 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b07d1-e717-40c1-a8a9-bfa81c288b5f service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:02.181407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b07d1-e717-40c1-a8a9-bfa81c288b5f service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:02.181722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314b07d1-e717-40c1-a8a9-bfa81c288b5f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:02.184089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-314b07d1-e717-40c1-a8a9-bfa81c288b5f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:02.215738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9829d4d-0783-492a-8e80-21a94a7dcbcd service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:02.230708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9829d4d-0783-492a-8e80-21a94a7dcbcd service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:02.243706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf6428-e565-4483-80f5-5bf18eb7f652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:02.245787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf6428-e565-4483-80f5-5bf18eb7f652 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:02.246080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf6428-e565-4483-80f5-5bf18eb7f652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:02.246362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bf6428-e565-4483-80f5-5bf18eb7f652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:02.248538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-83bf6428-e565-4483-80f5-5bf18eb7f652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:02.296435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b07d1-e717-40c1-a8a9-bfa81c288b5f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:02.297839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314b07d1-e717-40c1-a8a9-bfa81c288b5f service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:02.333873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1217/2436] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:59:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:59:02.348170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf6428-e565-4483-80f5-5bf18eb7f652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:02.349970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83bf6428-e565-4483-80f5-5bf18eb7f652 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:02.361038 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1220/2437] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:59:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:59:07.283289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b07d1-e717-40c1-a8a9-bfa81c288b5f service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:07.309043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314b07d1-e717-40c1-a8a9-bfa81c288b5f service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:07.328313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667f3bb-6875-44cd-971f-5e7c80b9ea91 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:07.329206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667f3bb-6875-44cd-971f-5e7c80b9ea91 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:07.329746 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667f3bb-6875-44cd-971f-5e7c80b9ea91 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:07.330965 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8667f3bb-6875-44cd-971f-5e7c80b9ea91 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:07.333262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8667f3bb-6875-44cd-971f-5e7c80b9ea91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:07.344199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf6428-e565-4483-80f5-5bf18eb7f652 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:07.365848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667f3bb-6875-44cd-971f-5e7c80b9ea91 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:07.367930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8667f3bb-6875-44cd-971f-5e7c80b9ea91 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:07.380205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bf6428-e565-4483-80f5-5bf18eb7f652 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:07.385958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60606a6-f42a-41ba-8cda-425b44dd8568 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:07.386837 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60606a6-f42a-41ba-8cda-425b44dd8568 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:07.387304 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60606a6-f42a-41ba-8cda-425b44dd8568 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:07.387734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c60606a6-f42a-41ba-8cda-425b44dd8568 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:07.389808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c60606a6-f42a-41ba-8cda-425b44dd8568 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:07.396010 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1218/2438] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:59:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:59:07.404314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60606a6-f42a-41ba-8cda-425b44dd8568 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:07.406257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c60606a6-f42a-41ba-8cda-425b44dd8568 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:07.419212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1221/2439] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:59:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 19:59:13.201967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667f3bb-6875-44cd-971f-5e7c80b9ea91 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:13.203415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8667f3bb-6875-44cd-971f-5e7c80b9ea91 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:13.233918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8667f3bb-6875-44cd-971f-5e7c80b9ea91 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:13.243888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:13.245244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:13.245792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:13.246295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] PATH_INFO: `/v3/users/8c67fe170c60476d88610d85f2528c52` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:13.268336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8c67fe170c60476d88610d85f2528c52)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:13.270936 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:13.343947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8c67fe170c60476d88610d85f2528c52'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.379421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8c67fe170c60476d88610d85f2528c52'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.387956 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8c67fe170c60476d88610d85f2528c52'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.389136 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8c67fe170c60476d88610d85f2528c52'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.389947 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8c67fe170c60476d88610d85f2528c52'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.407632 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1219/2440] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:59:13 2022] DELETE /identity/v3/users/8c67fe170c60476d88610d85f2528c52 => generated 0 bytes in 206 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:59:13.494994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60606a6-f42a-41ba-8cda-425b44dd8568 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:13.496753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c60606a6-f42a-41ba-8cda-425b44dd8568 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:13.517746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c60606a6-f42a-41ba-8cda-425b44dd8568 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:13.602137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:13.603186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:13.603571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:13.603950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] PATH_INFO: `/v3/users/4d7204f863c64eeeb7dab59a30c447b8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:13.622890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d7204f863c64eeeb7dab59a30c447b8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:13.624947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:13.656692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:13.658374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:13.674002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5839541b-51f9-41f3-af68-b5390249f202 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:13.680997 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:13.681829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:13.682116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:13.682665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] PATH_INFO: `/v3/projects/8064950f9a1942149aa8d6c08c206850` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:13.695270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:59:13.698598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8064950f9a1942149aa8d6c08c206850)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:13.700838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:13.708971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:59:13.715251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d7204f863c64eeeb7dab59a30c447b8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.738456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d7204f863c64eeeb7dab59a30c447b8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.758491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d7204f863c64eeeb7dab59a30c447b8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.760885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d7204f863c64eeeb7dab59a30c447b8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.762322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8064950f9a1942149aa8d6c08c206850', 'request_id': 'req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.762807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d7204f863c64eeeb7dab59a30c447b8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.768442 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8064950f9a1942149aa8d6c08c206850', 'request_id': 'req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.783394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8064950f9a1942149aa8d6c08c206850', 'request_id': 'req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.803451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] The token cache is being invalidate because project 8064950f9a1942149aa8d6c08c206850 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:59:13.804085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.804830 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:13.805231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1222/2441] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:59:13 2022] DELETE /identity/v3/users/4d7204f863c64eeeb7dab59a30c447b8 => generated 0 bytes in 310 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:59:13.809543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1220/2442] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:59:13 2022] DELETE /identity/v3/projects/8064950f9a1942149aa8d6c08c206850 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:59:13.810885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:13.813030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:13.900891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67b4bc46-98cc-4d89-9fb2-fb702fab11ca admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:13.906422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:13.907320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:13.907677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:13.907878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] PATH_INFO: `/v3/users/312a873d649e40fd9ec1949e4d077090` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:13.937919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] RBAC: Authorizing `identity:delete_user(user_id=312a873d649e40fd9ec1949e4d077090)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:13.939892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:13.993469 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '312a873d649e40fd9ec1949e4d077090'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.001207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:14.002740 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:14.012558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '312a873d649e40fd9ec1949e4d077090'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.019864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '312a873d649e40fd9ec1949e4d077090'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.020732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '312a873d649e40fd9ec1949e4d077090'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.021480 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '312a873d649e40fd9ec1949e4d077090'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.038681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1223/2443] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:59:13 2022] DELETE /identity/v3/users/312a873d649e40fd9ec1949e4d077090 => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:59:14.085908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3fd045-ddaf-4ad7-bbcd-162f390488b3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:14.148159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:14.149047 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:14.149331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:14.149643 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] PATH_INFO: `/v3/projects/da296fe635c84b99b1645ceaf0138b83` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:14.160722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da296fe635c84b99b1645ceaf0138b83)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:14.162732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:14.198001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:59:14.218153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da296fe635c84b99b1645ceaf0138b83', 'request_id': 'req-0c08f189-617f-4a46-a84f-15623b6d8469'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.225558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da296fe635c84b99b1645ceaf0138b83', 'request_id': 'req-0c08f189-617f-4a46-a84f-15623b6d8469'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.234118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da296fe635c84b99b1645ceaf0138b83', 'request_id': 'req-0c08f189-617f-4a46-a84f-15623b6d8469'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.234298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:14.236026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:14.249420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] The token cache is being invalidate because project da296fe635c84b99b1645ceaf0138b83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:59:14.249968 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.250982 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.253007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1221/2444] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:59:14 2022] DELETE /identity/v3/projects/da296fe635c84b99b1645ceaf0138b83 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:59:14.254298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df93c3a-0d98-478c-8ee1-3faf95ee0267 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:14.258009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:14.259427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:14.287664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:59:14.323787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:14.324589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:14.324876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:14.325151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] PATH_INFO: `/v3/projects/466bd0dde0234ba78083c98206c4f462` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:14.331528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c08f189-617f-4a46-a84f-15623b6d8469 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:14.336420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:14.336936 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=466bd0dde0234ba78083c98206c4f462)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:14.337305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:14.337627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:14.337912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] PATH_INFO: `/v3/users/166b54d7b99a432e931a10e2c1b71eb0` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:14.338904 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:14.352980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] RBAC: Authorizing `identity:delete_user(user_id=166b54d7b99a432e931a10e2c1b71eb0)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:14.355134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:14.390816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '466bd0dde0234ba78083c98206c4f462', 'request_id': 'req-34cfd208-07d1-4733-9b85-1ca72660d0e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.398317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '466bd0dde0234ba78083c98206c4f462', 'request_id': 'req-34cfd208-07d1-4733-9b85-1ca72660d0e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.403306 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '166b54d7b99a432e931a10e2c1b71eb0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.404765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '466bd0dde0234ba78083c98206c4f462', 'request_id': 'req-34cfd208-07d1-4733-9b85-1ca72660d0e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.418533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '166b54d7b99a432e931a10e2c1b71eb0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.419444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] The token cache is being invalidate because project 466bd0dde0234ba78083c98206c4f462 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:59:14.419739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.420355 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.422091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1224/2445] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:59:14 2022] DELETE /identity/v3/projects/466bd0dde0234ba78083c98206c4f462 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:59:14.426041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '166b54d7b99a432e931a10e2c1b71eb0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.426913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '166b54d7b99a432e931a10e2c1b71eb0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.427556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '166b54d7b99a432e931a10e2c1b71eb0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.443576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1222/2446] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 19:59:14 2022] DELETE /identity/v3/users/166b54d7b99a432e931a10e2c1b71eb0 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 19:59:14.649389 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:14.651659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:14.728718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34cfd208-07d1-4733-9b85-1ca72660d0e5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:14.794170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:59:14.800611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:14.801542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:14.802258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:14.802652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] PATH_INFO: `/v3/projects/dc3dc6e75b914c7c9cd8a664d7431d90` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:14.813485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc3dc6e75b914c7c9cd8a664d7431d90)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:14.815416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:14.865943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc3dc6e75b914c7c9cd8a664d7431d90', 'request_id': 'req-25b4d600-9c2d-4289-afb7-b28f276e896f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.873456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc3dc6e75b914c7c9cd8a664d7431d90', 'request_id': 'req-25b4d600-9c2d-4289-afb7-b28f276e896f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.879231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc3dc6e75b914c7c9cd8a664d7431d90', 'request_id': 'req-25b4d600-9c2d-4289-afb7-b28f276e896f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.894655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] The token cache is being invalidate because project dc3dc6e75b914c7c9cd8a664d7431d90 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 19:59:14.894946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.896193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 19:59:14.897778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1225/2447] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 19:59:14 2022] DELETE /identity/v3/projects/dc3dc6e75b914c7c9cd8a664d7431d90 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 19:59:24.439086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=72,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:59:24.891520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=29,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 19:59:58.421783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 19:59:58.423212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:58.495476 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00933030-cf9a-4632-8e04-d6570460f453 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 19:59:58.543696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c928b22-54d6-463f-a794-0e4b2738d410 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 19:59:58.544362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c928b22-54d6-463f-a794-0e4b2738d410 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 19:59:58.544644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c928b22-54d6-463f-a794-0e4b2738d410 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 19:59:58.544921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c928b22-54d6-463f-a794-0e4b2738d410 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 19:59:58.546710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0c928b22-54d6-463f-a794-0e4b2738d410 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 19:59:58.619528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c928b22-54d6-463f-a794-0e4b2738d410 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 19:59:58.620883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c928b22-54d6-463f-a794-0e4b2738d410 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 19:59:58.646622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1223/2448] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 19:59:58 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 225 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:00:08.642930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=81 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:00:11.290111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:00:11.292496 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:00:11.312207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b4d600-9c2d-4289-afb7-b28f276e896f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:00:11.316191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ed880-e37e-4d10-b75a-55af297e8767 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:00:11.316850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed880-e37e-4d10-b75a-55af297e8767 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:00:11.317132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed880-e37e-4d10-b75a-55af297e8767 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:00:11.317404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb7ed880-e37e-4d10-b75a-55af297e8767 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:00:11.319882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-eb7ed880-e37e-4d10-b75a-55af297e8767 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:00:11.393338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ed880-e37e-4d10-b75a-55af297e8767 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:00:11.395061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb7ed880-e37e-4d10-b75a-55af297e8767 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:00:11.415528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1226/2449] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:00:11 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:00:21.411903 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=36 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:00:55.759697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c928b22-54d6-463f-a794-0e4b2738d410 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:00:55.761027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0c928b22-54d6-463f-a794-0e4b2738d410 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:00:55.832726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c928b22-54d6-463f-a794-0e4b2738d410 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:00:55.836355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eba215-8a55-4efb-b6b2-2c13b50487d6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:00:55.837017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eba215-8a55-4efb-b6b2-2c13b50487d6 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:00:55.837295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eba215-8a55-4efb-b6b2-2c13b50487d6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:00:55.837610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83eba215-8a55-4efb-b6b2-2c13b50487d6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:00:55.839382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-83eba215-8a55-4efb-b6b2-2c13b50487d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:00:55.926420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eba215-8a55-4efb-b6b2-2c13b50487d6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:00:55.927868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83eba215-8a55-4efb-b6b2-2c13b50487d6 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:00:55.939768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1224/2450] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:00:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:01:00.872014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ed880-e37e-4d10-b75a-55af297e8767 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:00.882234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb7ed880-e37e-4d10-b75a-55af297e8767 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:00.885842 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17b36d-4927-48e0-89d8-05e0774839bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:00.886493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17b36d-4927-48e0-89d8-05e0774839bc service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:00.886775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17b36d-4927-48e0-89d8-05e0774839bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:00.887054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc17b36d-4927-48e0-89d8-05e0774839bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:00.888839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bc17b36d-4927-48e0-89d8-05e0774839bc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:00.956975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17b36d-4927-48e0-89d8-05e0774839bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:00.958455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc17b36d-4927-48e0-89d8-05e0774839bc service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:00.968438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1227/2451] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:01:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:01:04.258276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eba215-8a55-4efb-b6b2-2c13b50487d6 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:04.259829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-83eba215-8a55-4efb-b6b2-2c13b50487d6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:04.277555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83eba215-8a55-4efb-b6b2-2c13b50487d6 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:04.279222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:04.280113 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:04.280427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:04.281410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] PATH_INFO: `/v3/users/9192f2842ca044f08e5f95f98f2e1b88` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:04.282636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9192f2842ca044f08e5f95f98f2e1b88)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:04.284602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:04.336900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9192f2842ca044f08e5f95f98f2e1b88'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:01:04.349461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9192f2842ca044f08e5f95f98f2e1b88'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:01:04.356347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9192f2842ca044f08e5f95f98f2e1b88'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:01:04.357058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9192f2842ca044f08e5f95f98f2e1b88'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:01:04.357699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-5cf4c772-364d-4e8a-870b-01ccd1428612 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9192f2842ca044f08e5f95f98f2e1b88'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:01:04.360028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=9 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:01:04.371923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1225/2452] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:01:04 2022] DELETE /identity/v3/users/9192f2842ca044f08e5f95f98f2e1b88 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:01:04.550529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17b36d-4927-48e0-89d8-05e0774839bc service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:04.562641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc17b36d-4927-48e0-89d8-05e0774839bc service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:04.623086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:04.623940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:04.624224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:04.624536 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] PATH_INFO: `/v3/projects/8821facab6ed48588d9d484b5e3d98ba` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:04.625766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8821facab6ed48588d9d484b5e3d98ba)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:04.627681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:04.676573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8821facab6ed48588d9d484b5e3d98ba', 'request_id': 'req-33222c90-8647-47a2-b999-f878c5c86bcf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:01:04.682782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8821facab6ed48588d9d484b5e3d98ba', 'request_id': 'req-33222c90-8647-47a2-b999-f878c5c86bcf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:01:04.688736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8821facab6ed48588d9d484b5e3d98ba', 'request_id': 'req-33222c90-8647-47a2-b999-f878c5c86bcf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:01:04.703037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] The token cache is being invalidate because project 8821facab6ed48588d9d484b5e3d98ba was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:01:04.703414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:01:04.704133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-33222c90-8647-47a2-b999-f878c5c86bcf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:01:04.705806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1228/2453] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:01:04 2022] DELETE /identity/v3/projects/8821facab6ed48588d9d484b5e3d98ba => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:01:04.711881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d55082-bfd3-4c3b-9c17-50fcb3d34d73 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:04.712178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d55082-bfd3-4c3b-9c17-50fcb3d34d73 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:04.712456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d55082-bfd3-4c3b-9c17-50fcb3d34d73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:04.772340 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-28d55082-bfd3-4c3b-9c17-50fcb3d34d73 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:01:04.773592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-28d55082-bfd3-4c3b-9c17-50fcb3d34d73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:04.832527 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1226/2454] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:01:04.840609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130b68-2d2b-45cc-abd9-c8a884555d93 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:04.840929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130b68-2d2b-45cc-abd9-c8a884555d93 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:04.841214 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3130b68-2d2b-45cc-abd9-c8a884555d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:04.860999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a3130b68-2d2b-45cc-abd9-c8a884555d93 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:01:04.862257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a3130b68-2d2b-45cc-abd9-c8a884555d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:04.870324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1229/2455] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:01:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:01:04.874926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d55082-bfd3-4c3b-9c17-50fcb3d34d73 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:04.884772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d55082-bfd3-4c3b-9c17-50fcb3d34d73 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:04.889377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd07d8-0099-4747-bac9-76d6f2f90641 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:04.890635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd07d8-0099-4747-bac9-76d6f2f90641 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:04.890985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd07d8-0099-4747-bac9-76d6f2f90641 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:04.891273 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dafd07d8-0099-4747-bac9-76d6f2f90641 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:04.892002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd07d8-0099-4747-bac9-76d6f2f90641 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:01:04.892312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd07d8-0099-4747-bac9-76d6f2f90641 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:04.893975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dafd07d8-0099-4747-bac9-76d6f2f90641 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:04.905561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1227/2456] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:01:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:01:04.910518 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3130b68-2d2b-45cc-abd9-c8a884555d93 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:04.919973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3130b68-2d2b-45cc-abd9-c8a884555d93 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:04.925253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae35772-f0f9-4bbf-96e4-9b5e64aabbb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:04.925939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae35772-f0f9-4bbf-96e4-9b5e64aabbb7 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:04.926226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae35772-f0f9-4bbf-96e4-9b5e64aabbb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:04.926521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae35772-f0f9-4bbf-96e4-9b5e64aabbb7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:04.927260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae35772-f0f9-4bbf-96e4-9b5e64aabbb7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:04.929193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ae35772-f0f9-4bbf-96e4-9b5e64aabbb7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:04.944818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:01:04.968467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1230/2457] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:01:04 2022] POST /identity/v3/projects => generated 385 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:01:04.972814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd07d8-0099-4747-bac9-76d6f2f90641 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:04.983264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dafd07d8-0099-4747-bac9-76d6f2f90641 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:05.044616 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3a5a1-040a-4ce3-8e8e-58d451db5856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:05.045294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3a5a1-040a-4ce3-8e8e-58d451db5856 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:05.045620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3a5a1-040a-4ce3-8e8e-58d451db5856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:05.045885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e3a5a1-040a-4ce3-8e8e-58d451db5856 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:05.047200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3a5a1-040a-4ce3-8e8e-58d451db5856 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:05.049288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66e3a5a1-040a-4ce3-8e8e-58d451db5856 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:05.070300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1228/2458] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:01:04 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:01:05.074692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae35772-f0f9-4bbf-96e4-9b5e64aabbb7 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:05.085378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae35772-f0f9-4bbf-96e4-9b5e64aabbb7 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:05.090054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759faf87-1619-4285-9d51-91e2cacd449f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:05.090713 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759faf87-1619-4285-9d51-91e2cacd449f admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:05.091001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759faf87-1619-4285-9d51-91e2cacd449f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:05.091282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-759faf87-1619-4285-9d51-91e2cacd449f admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:05.092009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759faf87-1619-4285-9d51-91e2cacd449f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:05.094689 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-759faf87-1619-4285-9d51-91e2cacd449f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:05.105725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1231/2459] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:01:05.110751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3a5a1-040a-4ce3-8e8e-58d451db5856 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:05.121812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e3a5a1-040a-4ce3-8e8e-58d451db5856 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:05.127139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07ee3046-38ab-45d0-b31a-a39380be6622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:05.127811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee3046-38ab-45d0-b31a-a39380be6622 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:05.128094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee3046-38ab-45d0-b31a-a39380be6622 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:05.128366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07ee3046-38ab-45d0-b31a-a39380be6622 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:05.129088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee3046-38ab-45d0-b31a-a39380be6622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:05.130707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07ee3046-38ab-45d0-b31a-a39380be6622 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:05.141206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1229/2460] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:01:05 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:01:05.145404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759faf87-1619-4285-9d51-91e2cacd449f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:05.156317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-759faf87-1619-4285-9d51-91e2cacd449f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:05.160992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae158-9699-4f5e-b9ba-5900a6617bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:05.161947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae158-9699-4f5e-b9ba-5900a6617bc3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:05.162251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae158-9699-4f5e-b9ba-5900a6617bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:05.162544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ae158-9699-4f5e-b9ba-5900a6617bc3 admin admin] PATH_INFO: `/v3/projects/b2b576df300148ad8d8d7dd3f8335f8e/users/0d1cf9b55ce04cf997bf23ad73222814/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:05.178253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ae158-9699-4f5e-b9ba-5900a6617bc3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2b576df300148ad8d8d7dd3f8335f8e, user_id=0d1cf9b55ce04cf997bf23ad73222814, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:05.180838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ae158-9699-4f5e-b9ba-5900a6617bc3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:05.194221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1232/2461] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:01:05 2022] PUT /identity/v3/projects/b2b576df300148ad8d8d7dd3f8335f8e/users/0d1cf9b55ce04cf997bf23ad73222814/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:01:05.199283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98eb577-9901-48f0-8b66-4305e58b2e94 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:05.199576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98eb577-9901-48f0-8b66-4305e58b2e94 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:05.199886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b98eb577-9901-48f0-8b66-4305e58b2e94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:05.217999 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b98eb577-9901-48f0-8b66-4305e58b2e94 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:01:05.220040 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b98eb577-9901-48f0-8b66-4305e58b2e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:05.246624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:01:05.285099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1230/2462] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:01:05.294631 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae158-9699-4f5e-b9ba-5900a6617bc3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:05.297698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8b9ae158-9699-4f5e-b9ba-5900a6617bc3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:05.361647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ae158-9699-4f5e-b9ba-5900a6617bc3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:05.365805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5003e08-1a3e-475e-abae-a009f1979d6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:05.366418 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5003e08-1a3e-475e-abae-a009f1979d6b service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:05.366704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5003e08-1a3e-475e-abae-a009f1979d6b service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:05.366985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5003e08-1a3e-475e-abae-a009f1979d6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:05.376518 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5003e08-1a3e-475e-abae-a009f1979d6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:05.377988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5003e08-1a3e-475e-abae-a009f1979d6b service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:05.387528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1233/2463] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:01:09.237603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494e5b1-d9cc-4dee-91ad-57819a180a18 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:09.238094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494e5b1-d9cc-4dee-91ad-57819a180a18 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:09.238188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c494e5b1-d9cc-4dee-91ad-57819a180a18 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:09.275977 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c494e5b1-d9cc-4dee-91ad-57819a180a18 None None] MFA Rules not processed for user `0d1cf9b55ce04cf997bf23ad73222814`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:01:09.279966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c494e5b1-d9cc-4dee-91ad-57819a180a18 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:09.379589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1231/2464] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:01:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:01:09.400669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5003e08-1a3e-475e-abae-a009f1979d6b service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:09.402492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d5003e08-1a3e-475e-abae-a009f1979d6b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:09.500268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5003e08-1a3e-475e-abae-a009f1979d6b service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:09.506850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9db47e-2fac-4390-947b-265203f70b09 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:09.507606 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9db47e-2fac-4390-947b-265203f70b09 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:09.507962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9db47e-2fac-4390-947b-265203f70b09 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:09.508313 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9db47e-2fac-4390-947b-265203f70b09 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:09.519293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9db47e-2fac-4390-947b-265203f70b09 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:09.521014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9db47e-2fac-4390-947b-265203f70b09 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:09.532720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1234/2465] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:01:09 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:01:11.044610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494e5b1-d9cc-4dee-91ad-57819a180a18 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:11.057061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c494e5b1-d9cc-4dee-91ad-57819a180a18 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:11.061226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f53b03-2c19-4e76-a3ec-4b7197fb2bd1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:11.063265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f53b03-2c19-4e76-a3ec-4b7197fb2bd1 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:11.063690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f53b03-2c19-4e76-a3ec-4b7197fb2bd1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:11.064189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f53b03-2c19-4e76-a3ec-4b7197fb2bd1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:11.066899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-64f53b03-2c19-4e76-a3ec-4b7197fb2bd1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:11.079447 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f53b03-2c19-4e76-a3ec-4b7197fb2bd1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:11.081025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f53b03-2c19-4e76-a3ec-4b7197fb2bd1 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:11.090648 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1232/2466] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:01:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:01:11.210508 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e07c0-fbe0-46be-8c31-684d49a60250 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:11.211088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e07c0-fbe0-46be-8c31-684d49a60250 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:11.211088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234e07c0-fbe0-46be-8c31-684d49a60250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:11.230229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-234e07c0-fbe0-46be-8c31-684d49a60250 None None] MFA Rules not processed for user `0d1cf9b55ce04cf997bf23ad73222814`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:01:11.231565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-234e07c0-fbe0-46be-8c31-684d49a60250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:11.246454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1235/2467] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:01:11 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:01:11.784721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f53b03-2c19-4e76-a3ec-4b7197fb2bd1 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:11.793472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f53b03-2c19-4e76-a3ec-4b7197fb2bd1 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:11.798170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211aaa0-57a3-47e8-8c94-4877b9df9025 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:11.798840 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211aaa0-57a3-47e8-8c94-4877b9df9025 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:11.799114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211aaa0-57a3-47e8-8c94-4877b9df9025 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:11.799578 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9211aaa0-57a3-47e8-8c94-4877b9df9025 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:11.801443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-9211aaa0-57a3-47e8-8c94-4877b9df9025 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:11.812925 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211aaa0-57a3-47e8-8c94-4877b9df9025 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:11.814424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9211aaa0-57a3-47e8-8c94-4877b9df9025 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:11.826594 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1233/2468] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:01:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:01:12.294522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e07c0-fbe0-46be-8c31-684d49a60250 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:12.296823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-234e07c0-fbe0-46be-8c31-684d49a60250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:12.356434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:01:12.457324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234e07c0-fbe0-46be-8c31-684d49a60250 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:12.478589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420131e5-485e-41c0-a2cd-219cb2ca323b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:12.479628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420131e5-485e-41c0-a2cd-219cb2ca323b service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:12.480174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420131e5-485e-41c0-a2cd-219cb2ca323b service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:12.480867 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420131e5-485e-41c0-a2cd-219cb2ca323b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:12.483176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-420131e5-485e-41c0-a2cd-219cb2ca323b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:12.511470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420131e5-485e-41c0-a2cd-219cb2ca323b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:12.518163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420131e5-485e-41c0-a2cd-219cb2ca323b service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:12.545317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1236/2469] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:01:12 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 251 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:01:21.821620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=59 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:01:21.999381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211aaa0-57a3-47e8-8c94-4877b9df9025 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:01:22.015528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9211aaa0-57a3-47e8-8c94-4877b9df9025 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:01:22.021636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc01d4-f224-4318-ab72-5ff1480e782f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:01:22.022529 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc01d4-f224-4318-ab72-5ff1480e782f service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:01:22.022987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc01d4-f224-4318-ab72-5ff1480e782f service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:01:22.023445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecc01d4-f224-4318-ab72-5ff1480e782f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:01:22.025488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8ecc01d4-f224-4318-ab72-5ff1480e782f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:01:22.039764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc01d4-f224-4318-ab72-5ff1480e782f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:01:22.041351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecc01d4-f224-4318-ab72-5ff1480e782f service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:01:22.052630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1234/2470] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:01:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:01:22.533562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=17 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:01:32.047827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:02:11.332938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420131e5-485e-41c0-a2cd-219cb2ca323b service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:02:11.363085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420131e5-485e-41c0-a2cd-219cb2ca323b service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:02:11.368096 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e418d2e7-5394-4ed9-b321-8b656fc32059 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:02:11.368764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418d2e7-5394-4ed9-b321-8b656fc32059 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:02:11.369045 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418d2e7-5394-4ed9-b321-8b656fc32059 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:02:11.369318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e418d2e7-5394-4ed9-b321-8b656fc32059 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:02:11.371110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e418d2e7-5394-4ed9-b321-8b656fc32059 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:02:11.404670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418d2e7-5394-4ed9-b321-8b656fc32059 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:02:11.406288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e418d2e7-5394-4ed9-b321-8b656fc32059 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:02:11.416041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1237/2471] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:02:11 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:02:13.302739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc01d4-f224-4318-ab72-5ff1480e782f service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:02:13.304402 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8ecc01d4-f224-4318-ab72-5ff1480e782f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:02:13.391483 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecc01d4-f224-4318-ab72-5ff1480e782f service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:02:13.395050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942368a5-7ebe-4a4e-ac04-f96a52833540 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:02:13.395702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942368a5-7ebe-4a4e-ac04-f96a52833540 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:02:13.395984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942368a5-7ebe-4a4e-ac04-f96a52833540 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:02:13.396262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942368a5-7ebe-4a4e-ac04-f96a52833540 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:02:13.398073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-942368a5-7ebe-4a4e-ac04-f96a52833540 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:02:13.472653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942368a5-7ebe-4a4e-ac04-f96a52833540 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:02:13.474146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942368a5-7ebe-4a4e-ac04-f96a52833540 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:02:13.483921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1235/2472] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:02:13 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:02:21.411877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=22 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:02:23.478975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=61 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:02:52.409324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57b5b0-e05a-46e1-8771-6386c52813d8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:02:52.409927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57b5b0-e05a-46e1-8771-6386c52813d8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:02:52.409927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae57b5b0-e05a-46e1-8771-6386c52813d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:02:52.428050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ae57b5b0-e05a-46e1-8771-6386c52813d8 None None] MFA Rules not processed for user `0d1cf9b55ce04cf997bf23ad73222814`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:02:52.429307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ae57b5b0-e05a-46e1-8771-6386c52813d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:02:52.437002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1238/2473] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:02:52 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:02:55.574988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942368a5-7ebe-4a4e-ac04-f96a52833540 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:02:55.604604 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942368a5-7ebe-4a4e-ac04-f96a52833540 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:02:55.604604 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73332f-ce08-44c6-8b98-38532801d40f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:02:55.605167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73332f-ce08-44c6-8b98-38532801d40f service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:02:55.605465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73332f-ce08-44c6-8b98-38532801d40f service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:02:55.605881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73332f-ce08-44c6-8b98-38532801d40f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:02:55.607882 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-be73332f-ce08-44c6-8b98-38532801d40f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:02:55.630176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73332f-ce08-44c6-8b98-38532801d40f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:02:55.632178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73332f-ce08-44c6-8b98-38532801d40f service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:02:55.649575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1236/2474] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:02:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:03:02.422847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:03:05.641293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:03:10.121015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57b5b0-e05a-46e1-8771-6386c52813d8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:03:10.143122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae57b5b0-e05a-46e1-8771-6386c52813d8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:03:10.153825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14762-25c8-436e-94b9-9fc0f3c3fbf1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:03:10.154870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14762-25c8-436e-94b9-9fc0f3c3fbf1 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:03:10.155339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14762-25c8-436e-94b9-9fc0f3c3fbf1 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:03:10.155939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e14762-25c8-436e-94b9-9fc0f3c3fbf1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:03:10.158178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-81e14762-25c8-436e-94b9-9fc0f3c3fbf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:03:10.176623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e14762-25c8-436e-94b9-9fc0f3c3fbf1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:03:10.178494 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e14762-25c8-436e-94b9-9fc0f3c3fbf1 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:03:10.200719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1239/2475] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:03:10 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 80 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:03:20.196136 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:03:30.526075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73332f-ce08-44c6-8b98-38532801d40f service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:03:30.535521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73332f-ce08-44c6-8b98-38532801d40f service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:03:30.539195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6eb02-b9a8-4fe5-98cd-24822bd9bcd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:03:30.540505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6eb02-b9a8-4fe5-98cd-24822bd9bcd0 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:03:30.541276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6eb02-b9a8-4fe5-98cd-24822bd9bcd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:03:30.541446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a6eb02-b9a8-4fe5-98cd-24822bd9bcd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:03:30.543287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-03a6eb02-b9a8-4fe5-98cd-24822bd9bcd0 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:03:30.576240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6eb02-b9a8-4fe5-98cd-24822bd9bcd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:03:30.577701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03a6eb02-b9a8-4fe5-98cd-24822bd9bcd0 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:03:30.587823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1237/2476] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:03:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:03:31.100816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14762-25c8-436e-94b9-9fc0f3c3fbf1 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:03:31.102335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-81e14762-25c8-436e-94b9-9fc0f3c3fbf1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:03:31.139971 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e14762-25c8-436e-94b9-9fc0f3c3fbf1 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:03:31.144478 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:03:31.145331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:03:31.145645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:03:31.145921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] PATH_INFO: `/v3/users/78ae826a812b44328c48b82e9498ad8b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:03:31.147095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=78ae826a812b44328c48b82e9498ad8b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:03:31.148991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:03:31.195151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '78ae826a812b44328c48b82e9498ad8b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:03:31.207640 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '78ae826a812b44328c48b82e9498ad8b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:03:31.215464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '78ae826a812b44328c48b82e9498ad8b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:03:31.216166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '78ae826a812b44328c48b82e9498ad8b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:03:31.217066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '78ae826a812b44328c48b82e9498ad8b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:03:31.233629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1240/2477] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:03:31 2022] DELETE /identity/v3/users/78ae826a812b44328c48b82e9498ad8b => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:03:31.410498 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6eb02-b9a8-4fe5-98cd-24822bd9bcd0 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:03:31.421414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a6eb02-b9a8-4fe5-98cd-24822bd9bcd0 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:03:31.481809 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:03:31.482670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:03:31.482949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:03:31.483222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] PATH_INFO: `/v3/projects/e0bb4af78c1d4589b1d3cc90d5e6be63` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:03:31.484410 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=e0bb4af78c1d4589b1d3cc90d5e6be63)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:03:31.486350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:03:31.533494 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e0bb4af78c1d4589b1d3cc90d5e6be63', 'request_id': 'req-d1965871-db63-41c1-92e5-c295d78d04fa'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:03:31.539413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e0bb4af78c1d4589b1d3cc90d5e6be63', 'request_id': 'req-d1965871-db63-41c1-92e5-c295d78d04fa'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:03:31.546645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e0bb4af78c1d4589b1d3cc90d5e6be63', 'request_id': 'req-d1965871-db63-41c1-92e5-c295d78d04fa'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:03:31.558924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] The token cache is being invalidate because project e0bb4af78c1d4589b1d3cc90d5e6be63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:03:31.559194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:03:31.559844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d1965871-db63-41c1-92e5-c295d78d04fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:03:31.561649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1238/2478] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:03:31 2022] DELETE /identity/v3/projects/e0bb4af78c1d4589b1d3cc90d5e6be63 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:03:41.229703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=38,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:03:41.555342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=66,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:04:00.715076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:04:00.716764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:04:00.774145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f34ba6f-a9d4-4f02-b7d5-67feee8701f3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:04:00.822603 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e1bbec-7b09-4757-9962-21afc4dad781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:04:00.823258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1bbec-7b09-4757-9962-21afc4dad781 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:04:00.823538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1bbec-7b09-4757-9962-21afc4dad781 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:04:00.823811 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e1bbec-7b09-4757-9962-21afc4dad781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:04:00.825657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a0e1bbec-7b09-4757-9962-21afc4dad781 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:04:00.896337 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1bbec-7b09-4757-9962-21afc4dad781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:04:00.897881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e1bbec-7b09-4757-9962-21afc4dad781 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:04:00.906919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1241/2479] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:04:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:04:10.902700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=73 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:04:49.852477 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da6311-dfd5-482d-8bbf-d9527fd726d9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:04:49.853030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da6311-dfd5-482d-8bbf-d9527fd726d9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:04:49.853030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2da6311-dfd5-482d-8bbf-d9527fd726d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:04:49.870716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f2da6311-dfd5-482d-8bbf-d9527fd726d9 None None] MFA Rules not processed for user `0d1cf9b55ce04cf997bf23ad73222814`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:04:49.871923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f2da6311-dfd5-482d-8bbf-d9527fd726d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:04:49.927017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1239/2480] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:04:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:04:59.920221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=25 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:05:00.279357 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604e42-f485-4318-ba2a-25f083abfdda None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:05:00.279894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604e42-f485-4318-ba2a-25f083abfdda None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:05:00.279894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47604e42-f485-4318-ba2a-25f083abfdda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:05:00.340822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-47604e42-f485-4318-ba2a-25f083abfdda None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:05:00.342063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-47604e42-f485-4318-ba2a-25f083abfdda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:05:00.426638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1242/2481] 10.222.0.69 () {58 vars in 967 bytes} [Wed Jul 20 20:05:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:05:00.476859 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da6311-dfd5-482d-8bbf-d9527fd726d9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:05:00.478328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f2da6311-dfd5-482d-8bbf-d9527fd726d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:05:00.571717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2da6311-dfd5-482d-8bbf-d9527fd726d9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:05:00.577053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd20619a-3adf-48c6-995d-bbbd9047129a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:05:00.577749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20619a-3adf-48c6-995d-bbbd9047129a service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:05:00.578037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20619a-3adf-48c6-995d-bbbd9047129a service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:05:00.578314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd20619a-3adf-48c6-995d-bbbd9047129a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:05:00.587590 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20619a-3adf-48c6-995d-bbbd9047129a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:05:00.589207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd20619a-3adf-48c6-995d-bbbd9047129a service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:05:00.598351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1240/2482] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 20:05:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 20:05:02.074566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317edbee-76a9-4373-b3a6-8ec9716495d7 None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:05:02.075174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317edbee-76a9-4373-b3a6-8ec9716495d7 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:05:02.075297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317edbee-76a9-4373-b3a6-8ec9716495d7 None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:05:02.076830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1243/2483] 10.222.0.69 () {54 vars in 912 bytes} [Wed Jul 20 20:05:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 20:05:02.082333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6018d-6986-4e01-a276-ea3dcf19e184 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:05:02.082837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6018d-6986-4e01-a276-ea3dcf19e184 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:05:02.082977 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e6018d-6986-4e01-a276-ea3dcf19e184 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:05:02.122019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c2e6018d-6986-4e01-a276-ea3dcf19e184 None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:05:02.123264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c2e6018d-6986-4e01-a276-ea3dcf19e184 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:05:02.211059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1241/2484] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 20:05:02 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:05:02.234657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317edbee-76a9-4373-b3a6-8ec9716495d7 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:05:02.236370 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-317edbee-76a9-4373-b3a6-8ec9716495d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:05:02.255169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317edbee-76a9-4373-b3a6-8ec9716495d7 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:05:02.264845 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7bf47f-00df-4d1b-b74d-c7ac6a0b836e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:05:02.266057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf47f-00df-4d1b-b74d-c7ac6a0b836e service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:05:02.266574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf47f-00df-4d1b-b74d-c7ac6a0b836e service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:05:02.267082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7bf47f-00df-4d1b-b74d-c7ac6a0b836e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:05:02.281884 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bf47f-00df-4d1b-b74d-c7ac6a0b836e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:05:02.283677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7bf47f-00df-4d1b-b74d-c7ac6a0b836e service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:05:02.294652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1244/2485] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:05:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:05:05.462927 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7217c7-d7f0-4c7b-a175-bead1cf74f53 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:05:05.463552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7217c7-d7f0-4c7b-a175-bead1cf74f53 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:05:05.463552 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7217c7-d7f0-4c7b-a175-bead1cf74f53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:05:05.493042 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-3a7217c7-d7f0-4c7b-a175-bead1cf74f53 None None] MFA Rules not processed for user `343e480dd5ee4389ba430129cd7daaa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:05:05.494322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3a7217c7-d7f0-4c7b-a175-bead1cf74f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:05:05.539011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:05:05.560016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1242/2486] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:05:05 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:05:09.745363 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd76ae1-06e0-491c-9e32-4222cdf1dce4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:05:09.745927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd76ae1-06e0-491c-9e32-4222cdf1dce4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:05:09.746037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bd76ae1-06e0-491c-9e32-4222cdf1dce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:05:09.772668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3bd76ae1-06e0-491c-9e32-4222cdf1dce4 None None] MFA Rules not processed for user `4d08da0ef69e4cf2abc6c18bbbc51a2f`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:05:09.773918 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3bd76ae1-06e0-491c-9e32-4222cdf1dce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:05:09.780745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1245/2487] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:05:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:05:15.552313 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:05:19.769803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=64 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:06:16.672294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7217c7-d7f0-4c7b-a175-bead1cf74f53 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:06:16.673774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3a7217c7-d7f0-4c7b-a175-bead1cf74f53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:06:16.740628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7217c7-d7f0-4c7b-a175-bead1cf74f53 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:06:16.744243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8dac2-8983-4287-a9e1-21570bfb051e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:06:16.744904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8dac2-8983-4287-a9e1-21570bfb051e service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:06:16.745190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8dac2-8983-4287-a9e1-21570bfb051e service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:06:16.745484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f8dac2-8983-4287-a9e1-21570bfb051e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:06:16.747852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d7f8dac2-8983-4287-a9e1-21570bfb051e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:06:16.757738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8dac2-8983-4287-a9e1-21570bfb051e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:06:16.759173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f8dac2-8983-4287-a9e1-21570bfb051e service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:06:16.767929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1243/2488] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:06:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:06:26.764906 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=30 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:06:59.728465 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd76ae1-06e0-491c-9e32-4222cdf1dce4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:06:59.729943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3bd76ae1-06e0-491c-9e32-4222cdf1dce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:06:59.812744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bd76ae1-06e0-491c-9e32-4222cdf1dce4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:06:59.818990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b069c9-8ec4-4563-a5a9-a5741bd29d65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:06:59.819638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b069c9-8ec4-4563-a5a9-a5741bd29d65 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:06:59.819923 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b069c9-8ec4-4563-a5a9-a5741bd29d65 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:06:59.820201 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35b069c9-8ec4-4563-a5a9-a5741bd29d65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:06:59.822019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-35b069c9-8ec4-4563-a5a9-a5741bd29d65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:06:59.832581 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b069c9-8ec4-4563-a5a9-a5741bd29d65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:06:59.834378 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35b069c9-8ec4-4563-a5a9-a5741bd29d65 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:06:59.843434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1246/2489] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:06:59 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:07:01.244989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8dac2-8983-4287-a9e1-21570bfb051e service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:07:01.246409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d7f8dac2-8983-4287-a9e1-21570bfb051e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:01.267453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f8dac2-8983-4287-a9e1-21570bfb051e service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:07:01.271696 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b40200e-238d-4ff8-aa09-589e35a4aba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:07:01.272338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40200e-238d-4ff8-aa09-589e35a4aba4 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:01.272620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40200e-238d-4ff8-aa09-589e35a4aba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:01.272898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b40200e-238d-4ff8-aa09-589e35a4aba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:01.274660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4b40200e-238d-4ff8-aa09-589e35a4aba4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:01.288431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40200e-238d-4ff8-aa09-589e35a4aba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:07:01.289912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b40200e-238d-4ff8-aa09-589e35a4aba4 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:07:01.301157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1244/2490] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:07:01 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:07:09.839224 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=38 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:07:11.295915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=10 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:07:13.990374 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b069c9-8ec4-4563-a5a9-a5741bd29d65 service nova] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:07:13.991793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-35b069c9-8ec4-4563-a5a9-a5741bd29d65 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:14.006893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35b069c9-8ec4-4563-a5a9-a5741bd29d65 service nova] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:07:14.011826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94740b-9162-4f02-8561-714aa9267075 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:07:14.012593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94740b-9162-4f02-8561-714aa9267075 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:14.012880 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94740b-9162-4f02-8561-714aa9267075 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:14.013152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c94740b-9162-4f02-8561-714aa9267075 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:14.014974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1c94740b-9162-4f02-8561-714aa9267075 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:14.029825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94740b-9162-4f02-8561-714aa9267075 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:07:14.031566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c94740b-9162-4f02-8561-714aa9267075 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:07:14.044133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1247/2491] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:07:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:07:17.233674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00084838-df0c-464a-9e1c-0d408589f9be None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:17.234177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00084838-df0c-464a-9e1c-0d408589f9be None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:17.234294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00084838-df0c-464a-9e1c-0d408589f9be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:17.252685 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-00084838-df0c-464a-9e1c-0d408589f9be None None] MFA Rules not processed for user `0d1cf9b55ce04cf997bf23ad73222814`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:07:17.254677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-00084838-df0c-464a-9e1c-0d408589f9be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:17.261717 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1245/2492] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:07:17 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:07:17.574175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94740b-9162-4f02-8561-714aa9267075 service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:07:17.583427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c94740b-9162-4f02-8561-714aa9267075 service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:07:17.587176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1baa1-3391-42b6-b020-3ba8add68129 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:07:17.587809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1baa1-3391-42b6-b020-3ba8add68129 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:17.588088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1baa1-3391-42b6-b020-3ba8add68129 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:17.588364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e1baa1-3391-42b6-b020-3ba8add68129 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:17.590202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d2e1baa1-3391-42b6-b020-3ba8add68129 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:17.600658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1baa1-3391-42b6-b020-3ba8add68129 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:07:17.602105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2e1baa1-3391-42b6-b020-3ba8add68129 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:07:17.609958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1248/2493] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:07:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:07:18.020370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00084838-df0c-464a-9e1c-0d408589f9be None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:07:18.036030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00084838-df0c-464a-9e1c-0d408589f9be None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:07:18.041300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59ddd8-9102-4e43-a549-ac33c6841c85 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:07:18.041988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59ddd8-9102-4e43-a549-ac33c6841c85 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:18.042275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59ddd8-9102-4e43-a549-ac33c6841c85 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:18.042548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c59ddd8-9102-4e43-a549-ac33c6841c85 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:18.044434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1c59ddd8-9102-4e43-a549-ac33c6841c85 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:18.056700 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59ddd8-9102-4e43-a549-ac33c6841c85 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:07:18.058337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c59ddd8-9102-4e43-a549-ac33c6841c85 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:07:18.069090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1246/2494] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:07:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:07:18.708281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1baa1-3391-42b6-b020-3ba8add68129 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:07:18.721640 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e1baa1-3391-42b6-b020-3ba8add68129 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:07:18.728007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d833f6d-4856-4885-9f00-dacedab568a8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:07:18.728651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d833f6d-4856-4885-9f00-dacedab568a8 service nova] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:18.728938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d833f6d-4856-4885-9f00-dacedab568a8 service nova] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:18.729216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d833f6d-4856-4885-9f00-dacedab568a8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:18.731039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1d833f6d-4856-4885-9f00-dacedab568a8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:18.751334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d833f6d-4856-4885-9f00-dacedab568a8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:07:18.752956 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d833f6d-4856-4885-9f00-dacedab568a8 service nova] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:07:18.772726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1249/2495] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:07:18 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:07:27.706517 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59ddd8-9102-4e43-a549-ac33c6841c85 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:07:27.721523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c59ddd8-9102-4e43-a549-ac33c6841c85 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:07:27.727544 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca6d49-912b-4edc-a5d7-35af5ed736ac service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:07:27.728208 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca6d49-912b-4edc-a5d7-35af5ed736ac service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:27.728495 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca6d49-912b-4edc-a5d7-35af5ed736ac service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:27.728775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ca6d49-912b-4edc-a5d7-35af5ed736ac service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:27.730806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-61ca6d49-912b-4edc-a5d7-35af5ed736ac service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:27.747424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca6d49-912b-4edc-a5d7-35af5ed736ac service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:07:27.749089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ca6d49-912b-4edc-a5d7-35af5ed736ac service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:07:27.763655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1247/2496] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:07:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:07:28.769960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=26 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:07:37.775307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=15 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:07:50.164579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f04ef-aebc-46ff-b5a1-9174abfc10fd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:50.165133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f04ef-aebc-46ff-b5a1-9174abfc10fd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:50.165278 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7f04ef-aebc-46ff-b5a1-9174abfc10fd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:50.183141 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5e7f04ef-aebc-46ff-b5a1-9174abfc10fd None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:07:50.184346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5e7f04ef-aebc-46ff-b5a1-9174abfc10fd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:50.193062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1250/2497] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 20:07:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:07:50.243641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca6d49-912b-4edc-a5d7-35af5ed736ac service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:07:50.260056 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ca6d49-912b-4edc-a5d7-35af5ed736ac service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:07:50.265644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8003b2b-aa66-4757-a1f5-55ca6e444fda service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:07:50.266367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8003b2b-aa66-4757-a1f5-55ca6e444fda service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:50.266660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8003b2b-aa66-4757-a1f5-55ca6e444fda service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:50.267083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8003b2b-aa66-4757-a1f5-55ca6e444fda service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:50.277905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8003b2b-aa66-4757-a1f5-55ca6e444fda service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:07:50.279550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8003b2b-aa66-4757-a1f5-55ca6e444fda service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:07:50.289961 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1248/2498] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 20:07:50 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 20:07:56.101550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f04ef-aebc-46ff-b5a1-9174abfc10fd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:07:56.112108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7f04ef-aebc-46ff-b5a1-9174abfc10fd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:07:56.115881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5427e-039e-442b-8a2a-50ae6f9dd1d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:07:56.116545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5427e-039e-442b-8a2a-50ae6f9dd1d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:56.116916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5427e-039e-442b-8a2a-50ae6f9dd1d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:56.117195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5427e-039e-442b-8a2a-50ae6f9dd1d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:56.119012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-75f5427e-039e-442b-8a2a-50ae6f9dd1d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:56.184980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5427e-039e-442b-8a2a-50ae6f9dd1d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:07:56.186472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5427e-039e-442b-8a2a-50ae6f9dd1d3 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:07:56.197605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1251/2499] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:07:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:07:59.934378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8003b2b-aa66-4757-a1f5-55ca6e444fda service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:07:59.959924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8003b2b-aa66-4757-a1f5-55ca6e444fda service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:07:59.975854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffd0ad-f7e1-4dd7-9082-c1d44ecc88d1 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:07:59.976944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffd0ad-f7e1-4dd7-9082-c1d44ecc88d1 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:07:59.977284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffd0ad-f7e1-4dd7-9082-c1d44ecc88d1 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:07:59.978381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffd0ad-f7e1-4dd7-9082-c1d44ecc88d1 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:07:59.980292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ffffd0ad-f7e1-4dd7-9082-c1d44ecc88d1 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:07:59.994526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffd0ad-f7e1-4dd7-9082-c1d44ecc88d1 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:07:59.996149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffffd0ad-f7e1-4dd7-9082-c1d44ecc88d1 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:00.002363 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5427e-039e-442b-8a2a-50ae6f9dd1d3 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:00.003908 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-75f5427e-039e-442b-8a2a-50ae6f9dd1d3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:00.012712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1249/2500] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:07:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:00.019865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5427e-039e-442b-8a2a-50ae6f9dd1d3 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:00.025491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:00.027091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:00.027563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:00.028177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] PATH_INFO: `/v3/users/4d08da0ef69e4cf2abc6c18bbbc51a2f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:00.030078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d08da0ef69e4cf2abc6c18bbbc51a2f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:00.032528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:00.127683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d08da0ef69e4cf2abc6c18bbbc51a2f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:00.143008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d08da0ef69e4cf2abc6c18bbbc51a2f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:00.152892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d08da0ef69e4cf2abc6c18bbbc51a2f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:00.153770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d08da0ef69e4cf2abc6c18bbbc51a2f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:00.154696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-4a136dfe-17f7-4004-9d6f-2c3f745f847f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d08da0ef69e4cf2abc6c18bbbc51a2f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:00.189207 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1252/2501] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:00 2022] DELETE /identity/v3/users/4d08da0ef69e4cf2abc6c18bbbc51a2f => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:08:00.392051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffd0ad-f7e1-4dd7-9082-c1d44ecc88d1 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:00.421105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffffd0ad-f7e1-4dd7-9082-c1d44ecc88d1 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:00.504625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:00.505555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:00.505849 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:00.506133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] PATH_INFO: `/v3/projects/d0ab510f529d4442bd0e379056c4c3e5` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:00.508067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0ab510f529d4442bd0e379056c4c3e5)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:00.510138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:00.573061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0ab510f529d4442bd0e379056c4c3e5', 'request_id': 'req-dc221751-684d-44da-8ac0-b49d1e96a1ae'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:00.579095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0ab510f529d4442bd0e379056c4c3e5', 'request_id': 'req-dc221751-684d-44da-8ac0-b49d1e96a1ae'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:00.592270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0ab510f529d4442bd0e379056c4c3e5', 'request_id': 'req-dc221751-684d-44da-8ac0-b49d1e96a1ae'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:00.606538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] The token cache is being invalidate because project d0ab510f529d4442bd0e379056c4c3e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:00.606844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:00.607662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-dc221751-684d-44da-8ac0-b49d1e96a1ae admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:00.609289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1250/2502] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:00 2022] DELETE /identity/v3/projects/d0ab510f529d4442bd0e379056c4c3e5 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:00.614262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b12015-491a-4831-8c57-5f7f73f2bd9a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:00.614939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b12015-491a-4831-8c57-5f7f73f2bd9a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:00.615443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b12015-491a-4831-8c57-5f7f73f2bd9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:00.636257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-24b12015-491a-4831-8c57-5f7f73f2bd9a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:00.637666 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-24b12015-491a-4831-8c57-5f7f73f2bd9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:00.700626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:00.717942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1253/2503] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:00.720951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c442ae-786a-4922-b796-717852dc0e0c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:00.720951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c442ae-786a-4922-b796-717852dc0e0c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:00.720951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c442ae-786a-4922-b796-717852dc0e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:00.741187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b2c442ae-786a-4922-b796-717852dc0e0c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:00.742657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b2c442ae-786a-4922-b796-717852dc0e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:00.754720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1251/2504] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:00.755954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b12015-491a-4831-8c57-5f7f73f2bd9a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:00.768758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b12015-491a-4831-8c57-5f7f73f2bd9a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:00.774517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f537d-0b40-45ac-a874-8590334e39fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:00.775186 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f537d-0b40-45ac-a874-8590334e39fa admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:00.775470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f537d-0b40-45ac-a874-8590334e39fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:00.775746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79f537d-0b40-45ac-a874-8590334e39fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:00.776475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f537d-0b40-45ac-a874-8590334e39fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:08:00.776785 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f537d-0b40-45ac-a874-8590334e39fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:00.778427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d79f537d-0b40-45ac-a874-8590334e39fa admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:00.793179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1254/2505] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:08:00.797665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c442ae-786a-4922-b796-717852dc0e0c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:00.808627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c442ae-786a-4922-b796-717852dc0e0c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:00.813531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a82646-afd3-444e-84bc-a1fb32c9e645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:00.814536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a82646-afd3-444e-84bc-a1fb32c9e645 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:00.814861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a82646-afd3-444e-84bc-a1fb32c9e645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:00.815137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86a82646-afd3-444e-84bc-a1fb32c9e645 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:00.815922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a82646-afd3-444e-84bc-a1fb32c9e645 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:00.819463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86a82646-afd3-444e-84bc-a1fb32c9e645 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:00.851708 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1252/2506] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:00 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:08:00.855921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f537d-0b40-45ac-a874-8590334e39fa admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:00.866952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79f537d-0b40-45ac-a874-8590334e39fa admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:00.936541 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c43e2c-b03b-4a28-beac-213623e3ad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:00.937244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c43e2c-b03b-4a28-beac-213623e3ad9a admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:00.937558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c43e2c-b03b-4a28-beac-213623e3ad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:00.937840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c43e2c-b03b-4a28-beac-213623e3ad9a admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:00.938574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c43e2c-b03b-4a28-beac-213623e3ad9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:00.940672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30c43e2c-b03b-4a28-beac-213623e3ad9a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:00.968281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1255/2507] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:00 2022] POST /identity/v3/users => generated 287 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:00.969992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a82646-afd3-444e-84bc-a1fb32c9e645 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:00.982614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86a82646-afd3-444e-84bc-a1fb32c9e645 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:00.987624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577a2ff-3916-4823-bfcd-1cbcf27da656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:00.988332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577a2ff-3916-4823-bfcd-1cbcf27da656 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:00.988619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577a2ff-3916-4823-bfcd-1cbcf27da656 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:00.988900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d577a2ff-3916-4823-bfcd-1cbcf27da656 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:00.989699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577a2ff-3916-4823-bfcd-1cbcf27da656 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:00.991905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d577a2ff-3916-4823-bfcd-1cbcf27da656 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:01.003851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1253/2508] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:01.007738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c43e2c-b03b-4a28-beac-213623e3ad9a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:01.018236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c43e2c-b03b-4a28-beac-213623e3ad9a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:01.023900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b663ce-e03f-4f2c-ade4-081459875c90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:01.024654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b663ce-e03f-4f2c-ade4-081459875c90 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:01.025009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b663ce-e03f-4f2c-ade4-081459875c90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:01.025381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b663ce-e03f-4f2c-ade4-081459875c90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:01.026236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b663ce-e03f-4f2c-ade4-081459875c90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:01.028987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b663ce-e03f-4f2c-ade4-081459875c90 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:01.064905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1256/2509] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:01.066700 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577a2ff-3916-4823-bfcd-1cbcf27da656 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:01.082728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d577a2ff-3916-4823-bfcd-1cbcf27da656 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:01.087914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea183539-34c6-4370-94cb-66e0e7a2c8c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:01.088800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea183539-34c6-4370-94cb-66e0e7a2c8c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:01.089084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea183539-34c6-4370-94cb-66e0e7a2c8c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:01.089360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea183539-34c6-4370-94cb-66e0e7a2c8c1 admin admin] PATH_INFO: `/v3/projects/6e2ed32e0b2745ceb5e6a01bb21b3ac6/users/9cda52391eb44481811d8578ef7b13a8/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:01.105857 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea183539-34c6-4370-94cb-66e0e7a2c8c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e2ed32e0b2745ceb5e6a01bb21b3ac6, user_id=9cda52391eb44481811d8578ef7b13a8, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:01.108623 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea183539-34c6-4370-94cb-66e0e7a2c8c1 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:01.119243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:01.125296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1254/2510] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:01 2022] PUT /identity/v3/projects/6e2ed32e0b2745ceb5e6a01bb21b3ac6/users/9cda52391eb44481811d8578ef7b13a8/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:08:01.135460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35088b9-173e-43fb-b5fe-e12b429e095f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:01.135827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35088b9-173e-43fb-b5fe-e12b429e095f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:01.136183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d35088b9-173e-43fb-b5fe-e12b429e095f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:01.171644 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d35088b9-173e-43fb-b5fe-e12b429e095f None None] MFA Rules not processed for user `9cda52391eb44481811d8578ef7b13a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:01.172877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d35088b9-173e-43fb-b5fe-e12b429e095f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:01.261364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1257/2511] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:01.274520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea183539-34c6-4370-94cb-66e0e7a2c8c1 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:01.276291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ea183539-34c6-4370-94cb-66e0e7a2c8c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:01.387871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea183539-34c6-4370-94cb-66e0e7a2c8c1 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:01.394385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27049155-1bb9-493a-bbd7-f028e5746bfc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:01.395055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27049155-1bb9-493a-bbd7-f028e5746bfc service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:01.395344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27049155-1bb9-493a-bbd7-f028e5746bfc service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:01.395625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27049155-1bb9-493a-bbd7-f028e5746bfc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:01.405901 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27049155-1bb9-493a-bbd7-f028e5746bfc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:01.409355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27049155-1bb9-493a-bbd7-f028e5746bfc service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:01.427137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1255/2512] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:01.860676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35088b9-173e-43fb-b5fe-e12b429e095f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:01.922040 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:01.928632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d35088b9-173e-43fb-b5fe-e12b429e095f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:01.933450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:01.934346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:01.934623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:01.934896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] PATH_INFO: `/v3/users/9cda52391eb44481811d8578ef7b13a8` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:01.936063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cda52391eb44481811d8578ef7b13a8)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:01.938004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:01.985485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cda52391eb44481811d8578ef7b13a8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:01.997686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cda52391eb44481811d8578ef7b13a8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:02.004427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cda52391eb44481811d8578ef7b13a8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:02.005115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cda52391eb44481811d8578ef7b13a8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:02.005745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cda52391eb44481811d8578ef7b13a8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:02.018981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1258/2513] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:01 2022] DELETE /identity/v3/users/9cda52391eb44481811d8578ef7b13a8 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:02.024310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1eb93-1dff-4416-a2d9-48925121c5af None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.024767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1eb93-1dff-4416-a2d9-48925121c5af None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.024885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c1eb93-1dff-4416-a2d9-48925121c5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.042397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f3c1eb93-1dff-4416-a2d9-48925121c5af None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:02.043597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f3c1eb93-1dff-4416-a2d9-48925121c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:02.102519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1256/2514] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:02.112413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:02.113874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:02.171222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c744403-6792-426a-9a57-5985d21d421c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:02.174942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0973965-f3d8-4e58-ada1-ab8b47cab130 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:02.175582 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0973965-f3d8-4e58-ada1-ab8b47cab130 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.175870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0973965-f3d8-4e58-ada1-ab8b47cab130 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.176152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0973965-f3d8-4e58-ada1-ab8b47cab130 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.185338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0973965-f3d8-4e58-ada1-ab8b47cab130 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:02.186804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0973965-f3d8-4e58-ada1-ab8b47cab130 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:02.196934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1259/2515] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:02.463880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1eb93-1dff-4416-a2d9-48925121c5af None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:02.465318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f3c1eb93-1dff-4416-a2d9-48925121c5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:02.477626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c1eb93-1dff-4416-a2d9-48925121c5af None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:02.482229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:02.483094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.483380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.483655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] PATH_INFO: `/v3/projects/6e2ed32e0b2745ceb5e6a01bb21b3ac6` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.484862 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e2ed32e0b2745ceb5e6a01bb21b3ac6)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:02.486858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:02.535599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e2ed32e0b2745ceb5e6a01bb21b3ac6', 'request_id': 'req-a3ed8be6-d028-445b-ae3c-894b922b0300'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:02.541205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e2ed32e0b2745ceb5e6a01bb21b3ac6', 'request_id': 'req-a3ed8be6-d028-445b-ae3c-894b922b0300'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:02.548438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e2ed32e0b2745ceb5e6a01bb21b3ac6', 'request_id': 'req-a3ed8be6-d028-445b-ae3c-894b922b0300'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:02.561006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] The token cache is being invalidate because project 6e2ed32e0b2745ceb5e6a01bb21b3ac6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:02.561290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:02.561934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-a3ed8be6-d028-445b-ae3c-894b922b0300 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:02.563457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1257/2516] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:02 2022] DELETE /identity/v3/projects/6e2ed32e0b2745ceb5e6a01bb21b3ac6 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:02.568408 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0f866-1780-4084-93f7-bd1cb8b063b0 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.568753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0f866-1780-4084-93f7-bd1cb8b063b0 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.569041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda0f866-1780-4084-93f7-bd1cb8b063b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.587495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-cda0f866-1780-4084-93f7-bd1cb8b063b0 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:02.588714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cda0f866-1780-4084-93f7-bd1cb8b063b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:02.648204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1260/2517] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:02.656069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736b07-5cec-4dfc-b6e9-1f57b82f713f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.656359 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736b07-5cec-4dfc-b6e9-1f57b82f713f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.656638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736b07-5cec-4dfc-b6e9-1f57b82f713f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.674658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-98736b07-5cec-4dfc-b6e9-1f57b82f713f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:02.675846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-98736b07-5cec-4dfc-b6e9-1f57b82f713f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:02.685074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1258/2518] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:02.689371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0f866-1780-4084-93f7-bd1cb8b063b0 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:02.701414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda0f866-1780-4084-93f7-bd1cb8b063b0 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:02.705006 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95816141-fbb5-48a6-ab83-7cbf73551120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:02.705707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95816141-fbb5-48a6-ab83-7cbf73551120 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.705997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95816141-fbb5-48a6-ab83-7cbf73551120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.706279 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95816141-fbb5-48a6-ab83-7cbf73551120 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.707031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95816141-fbb5-48a6-ab83-7cbf73551120 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:08:02.707338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95816141-fbb5-48a6-ab83-7cbf73551120 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:02.708934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95816141-fbb5-48a6-ab83-7cbf73551120 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:02.721417 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1261/2519] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:08:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:08:02.726672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736b07-5cec-4dfc-b6e9-1f57b82f713f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:02.731194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:02.738119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736b07-5cec-4dfc-b6e9-1f57b82f713f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:02.744904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9787a8-bd1a-42b7-a59c-3d21bbe69a84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:02.745610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9787a8-bd1a-42b7-a59c-3d21bbe69a84 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.745900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9787a8-bd1a-42b7-a59c-3d21bbe69a84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.746179 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9787a8-bd1a-42b7-a59c-3d21bbe69a84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.746889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9787a8-bd1a-42b7-a59c-3d21bbe69a84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:02.748820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9787a8-bd1a-42b7-a59c-3d21bbe69a84 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:02.775339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1259/2520] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:02 2022] POST /identity/v3/projects => generated 399 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:08:02.779615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95816141-fbb5-48a6-ab83-7cbf73551120 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:02.786076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:02.790844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95816141-fbb5-48a6-ab83-7cbf73551120 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:02.858195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75836f-390e-4ac4-9ed3-5cdea26d9702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:02.858876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75836f-390e-4ac4-9ed3-5cdea26d9702 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.859163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75836f-390e-4ac4-9ed3-5cdea26d9702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.859446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c75836f-390e-4ac4-9ed3-5cdea26d9702 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.860150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75836f-390e-4ac4-9ed3-5cdea26d9702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:02.862105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c75836f-390e-4ac4-9ed3-5cdea26d9702 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:02.881341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1262/2521] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:02 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:02.885542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9787a8-bd1a-42b7-a59c-3d21bbe69a84 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:02.895904 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9787a8-bd1a-42b7-a59c-3d21bbe69a84 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:02.900795 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82631d1-f854-437b-b3d2-8fa590415b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:02.901458 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82631d1-f854-437b-b3d2-8fa590415b2d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.901802 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82631d1-f854-437b-b3d2-8fa590415b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.902081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82631d1-f854-437b-b3d2-8fa590415b2d admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.902812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82631d1-f854-437b-b3d2-8fa590415b2d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:02.904416 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82631d1-f854-437b-b3d2-8fa590415b2d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:02.916382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1260/2522] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:02.920415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75836f-390e-4ac4-9ed3-5cdea26d9702 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:02.931244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c75836f-390e-4ac4-9ed3-5cdea26d9702 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:02.935958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254f76d6-a525-4095-98a1-99acfd07cacd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:02.936637 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f76d6-a525-4095-98a1-99acfd07cacd admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.936924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f76d6-a525-4095-98a1-99acfd07cacd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.937203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254f76d6-a525-4095-98a1-99acfd07cacd admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.937966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f76d6-a525-4095-98a1-99acfd07cacd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:02.939554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254f76d6-a525-4095-98a1-99acfd07cacd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:02.949824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1263/2523] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:02.954100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82631d1-f854-437b-b3d2-8fa590415b2d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:02.964198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82631d1-f854-437b-b3d2-8fa590415b2d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:02.968823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d340c-1079-4102-a700-4b4526185547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:02.969989 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d340c-1079-4102-a700-4b4526185547 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:02.970271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d340c-1079-4102-a700-4b4526185547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:02.970551 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7d340c-1079-4102-a700-4b4526185547 admin admin] PATH_INFO: `/v3/projects/41a9a438a6954d91b926467901f02f7c/users/cc38506d39ff4fe0841f032820a170f0/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:02.986154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d340c-1079-4102-a700-4b4526185547 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41a9a438a6954d91b926467901f02f7c, user_id=cc38506d39ff4fe0841f032820a170f0, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:02.988823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd7d340c-1079-4102-a700-4b4526185547 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:03.002990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1261/2524] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:02 2022] PUT /identity/v3/projects/41a9a438a6954d91b926467901f02f7c/users/cc38506d39ff4fe0841f032820a170f0/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:03.014651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1450409-10ce-4d34-bfe3-8338e0fdd54d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:03.014942 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1450409-10ce-4d34-bfe3-8338e0fdd54d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:03.015226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1450409-10ce-4d34-bfe3-8338e0fdd54d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:03.042251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a1450409-10ce-4d34-bfe3-8338e0fdd54d None None] MFA Rules not processed for user `cc38506d39ff4fe0841f032820a170f0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:03.043471 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a1450409-10ce-4d34-bfe3-8338e0fdd54d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:03.111491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1264/2525] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:03.116244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d340c-1079-4102-a700-4b4526185547 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:03.183332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7d340c-1079-4102-a700-4b4526185547 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:03.188018 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ee585-41ac-40e3-b7f2-629a5283b483 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:03.188689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ee585-41ac-40e3-b7f2-629a5283b483 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:03.188976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ee585-41ac-40e3-b7f2-629a5283b483 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:03.189256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1ee585-41ac-40e3-b7f2-629a5283b483 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:03.189977 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ee585-41ac-40e3-b7f2-629a5283b483 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:03.191902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1ee585-41ac-40e3-b7f2-629a5283b483 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:03.221098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1262/2526] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:03 2022] POST /identity/v3/projects => generated 399 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:03.222562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1450409-10ce-4d34-bfe3-8338e0fdd54d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:03.234757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1450409-10ce-4d34-bfe3-8338e0fdd54d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:03.294734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036c9e9-d226-4b98-9f33-9ba561be1025 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:03.295456 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c9e9-d226-4b98-9f33-9ba561be1025 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:03.295741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c9e9-d226-4b98-9f33-9ba561be1025 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:03.296045 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a036c9e9-d226-4b98-9f33-9ba561be1025 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:03.296754 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036c9e9-d226-4b98-9f33-9ba561be1025 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:03.298748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a036c9e9-d226-4b98-9f33-9ba561be1025 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:03.324995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1265/2527] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:03 2022] POST /identity/v3/users => generated 306 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:03.325726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ee585-41ac-40e3-b7f2-629a5283b483 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:03.337319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1ee585-41ac-40e3-b7f2-629a5283b483 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:03.342197 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddceec-ed9a-4a39-adaf-225be5698f7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:03.342808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddceec-ed9a-4a39-adaf-225be5698f7a admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:03.343095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddceec-ed9a-4a39-adaf-225be5698f7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:03.343375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ddceec-ed9a-4a39-adaf-225be5698f7a admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:03.344107 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ddceec-ed9a-4a39-adaf-225be5698f7a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:03.346218 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ddceec-ed9a-4a39-adaf-225be5698f7a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:03.357825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1263/2528] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:03.362175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036c9e9-d226-4b98-9f33-9ba561be1025 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:03.367796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:03.373546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a036c9e9-d226-4b98-9f33-9ba561be1025 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:03.380011 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873e7f4-a6b3-4e04-a724-4ca9a2d830e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:03.380896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873e7f4-a6b3-4e04-a724-4ca9a2d830e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:03.381181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873e7f4-a6b3-4e04-a724-4ca9a2d830e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:03.381457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3873e7f4-a6b3-4e04-a724-4ca9a2d830e3 admin admin] PATH_INFO: `/v3/projects/b782b4506e4942aba391a3bc2379820d/users/c9db7dc1a6744481b00dd54a9de69f75/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:03.398466 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873e7f4-a6b3-4e04-a724-4ca9a2d830e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b782b4506e4942aba391a3bc2379820d, user_id=c9db7dc1a6744481b00dd54a9de69f75, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:03.401176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3873e7f4-a6b3-4e04-a724-4ca9a2d830e3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:03.415627 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1266/2529] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:03 2022] PUT /identity/v3/projects/b782b4506e4942aba391a3bc2379820d/users/c9db7dc1a6744481b00dd54a9de69f75/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:03.419729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddceec-ed9a-4a39-adaf-225be5698f7a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:03.430095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ddceec-ed9a-4a39-adaf-225be5698f7a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:03.488029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82583a4-0947-4fc0-a8da-41502b0bd124 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:03.488707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82583a4-0947-4fc0-a8da-41502b0bd124 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:03.488991 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82583a4-0947-4fc0-a8da-41502b0bd124 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:03.489269 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82583a4-0947-4fc0-a8da-41502b0bd124 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:03.490035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82583a4-0947-4fc0-a8da-41502b0bd124 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:03.491908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82583a4-0947-4fc0-a8da-41502b0bd124 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:03.501994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1264/2530] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:03.506232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873e7f4-a6b3-4e04-a724-4ca9a2d830e3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:03.517588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3873e7f4-a6b3-4e04-a724-4ca9a2d830e3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:03.523221 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f647511-72db-46d7-b317-30fcd72510ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:03.524116 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f647511-72db-46d7-b317-30fcd72510ab admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:03.524402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f647511-72db-46d7-b317-30fcd72510ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:03.524704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f647511-72db-46d7-b317-30fcd72510ab admin admin] PATH_INFO: `/v3/projects/b782b4506e4942aba391a3bc2379820d/users/c9db7dc1a6744481b00dd54a9de69f75/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:03.526775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f647511-72db-46d7-b317-30fcd72510ab admin admin] RBAC: Authorizing `identity:create_grant(project_id=b782b4506e4942aba391a3bc2379820d, user_id=c9db7dc1a6744481b00dd54a9de69f75, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:03.529473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f647511-72db-46d7-b317-30fcd72510ab admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:03.539945 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1267/2531] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:03 2022] PUT /identity/v3/projects/b782b4506e4942aba391a3bc2379820d/users/c9db7dc1a6744481b00dd54a9de69f75/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:03.550228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1984c-6e82-43d8-9900-ac7acc6386b5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:03.550525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1984c-6e82-43d8-9900-ac7acc6386b5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:03.550810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41a1984c-6e82-43d8-9900-ac7acc6386b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:03.579351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-41a1984c-6e82-43d8-9900-ac7acc6386b5 None None] MFA Rules not processed for user `c9db7dc1a6744481b00dd54a9de69f75`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:03.580606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-41a1984c-6e82-43d8-9900-ac7acc6386b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:03.584760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:03.654579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1265/2532] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:03 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:03.665849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f647511-72db-46d7-b317-30fcd72510ab admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:03.667568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5f647511-72db-46d7-b317-30fcd72510ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:03.725647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f647511-72db-46d7-b317-30fcd72510ab admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:03.729341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-574f4152-5b63-42f5-a799-891ce85ea181 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:03.730012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f4152-5b63-42f5-a799-891ce85ea181 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:03.730289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f4152-5b63-42f5-a799-891ce85ea181 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:03.730559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-574f4152-5b63-42f5-a799-891ce85ea181 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:03.739450 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f4152-5b63-42f5-a799-891ce85ea181 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:03.740864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-574f4152-5b63-42f5-a799-891ce85ea181 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:03.750393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1268/2533] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:03 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:03.823126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1984c-6e82-43d8-9900-ac7acc6386b5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:03.832758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41a1984c-6e82-43d8-9900-ac7acc6386b5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:03.836839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d95c65-3b2f-4785-884b-b976c987b7a2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:03.837477 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d95c65-3b2f-4785-884b-b976c987b7a2 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:03.837790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d95c65-3b2f-4785-884b-b976c987b7a2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:03.838061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d95c65-3b2f-4785-884b-b976c987b7a2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:03.899005 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d95c65-3b2f-4785-884b-b976c987b7a2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:03.900467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d95c65-3b2f-4785-884b-b976c987b7a2 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:03.910011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1266/2534] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:03 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:07.223831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153309d6-325c-4fd1-a11a-7f2685bcd67a None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:07.223831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153309d6-325c-4fd1-a11a-7f2685bcd67a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:07.224566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153309d6-325c-4fd1-a11a-7f2685bcd67a None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:07.225651 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1269/2535] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 20:08:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 20:08:07.232427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878a18-3cd2-40e8-a32f-22a2c7656db3 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:07.232943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878a18-3cd2-40e8-a32f-22a2c7656db3 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:07.233105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d878a18-3cd2-40e8-a32f-22a2c7656db3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:07.270124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6d878a18-3cd2-40e8-a32f-22a2c7656db3 None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:07.271248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6d878a18-3cd2-40e8-a32f-22a2c7656db3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:07.341130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1267/2536] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 20:08:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:07.356134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153309d6-325c-4fd1-a11a-7f2685bcd67a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:07.357895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-153309d6-325c-4fd1-a11a-7f2685bcd67a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:07.436716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153309d6-325c-4fd1-a11a-7f2685bcd67a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:07.440266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e43219fc-47c3-4266-87e3-0e7f5e931762 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e46ec9c31094d03b6a413e09bc9f699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:07.440900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43219fc-47c3-4266-87e3-0e7f5e931762 service glance] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:07.441179 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43219fc-47c3-4266-87e3-0e7f5e931762 service glance] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:07.441449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43219fc-47c3-4266-87e3-0e7f5e931762 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:07.451455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43219fc-47c3-4266-87e3-0e7f5e931762 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:07.452866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e43219fc-47c3-4266-87e3-0e7f5e931762 service glance] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:07.463645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1270/2537] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:07.531103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de387-c5ee-49c0-be90-b2b6fb2e502a None None] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:07.531467 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de387-c5ee-49c0-be90-b2b6fb2e502a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:07.531814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86de387-c5ee-49c0-be90-b2b6fb2e502a None None] PATH_INFO: `/` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:07.533278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1268/2538] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 20:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 20:08:07.538192 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2edf0d-3bc5-44ba-ab1c-aacfc64c2cad None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:07.539070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2edf0d-3bc5-44ba-ab1c-aacfc64c2cad None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:07.539575 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2edf0d-3bc5-44ba-ab1c-aacfc64c2cad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:07.576015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ed2edf0d-3bc5-44ba-ab1c-aacfc64c2cad None None] MFA Rules not processed for user `3a1a88089d384964bd9774eda4c5d8a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:07.577217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ed2edf0d-3bc5-44ba-ab1c-aacfc64c2cad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:07.616296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:07.638479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1271/2539] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 20:08:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:07.650709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86de387-c5ee-49c0-be90-b2b6fb2e502a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:07.652425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d86de387-c5ee-49c0-be90-b2b6fb2e502a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:07.666155 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86de387-c5ee-49c0-be90-b2b6fb2e502a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:07.674568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ca470a-14d9-4370-8610-0b1bc75ad0d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:07.675348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca470a-14d9-4370-8610-0b1bc75ad0d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:07.675638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca470a-14d9-4370-8610-0b1bc75ad0d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:07.675921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ca470a-14d9-4370-8610-0b1bc75ad0d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:07.685046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca470a-14d9-4370-8610-0b1bc75ad0d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:07.686659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29ca470a-14d9-4370-8610-0b1bc75ad0d0 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:07.695331 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1269/2540] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:07.781764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92319-88e1-474f-a1c1-49e874887b7c None None] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:07.782203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92319-88e1-474f-a1c1-49e874887b7c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:07.782479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af92319-88e1-474f-a1c1-49e874887b7c None None] PATH_INFO: `/` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:07.783861 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1272/2541] 10.222.0.69 () {54 vars in 921 bytes} [Wed Jul 20 20:08:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 20 20:08:07.790491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622fb46-0ed6-4b9d-8a7b-ca0604deaabd None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:07.790783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622fb46-0ed6-4b9d-8a7b-ca0604deaabd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:07.791084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e622fb46-0ed6-4b9d-8a7b-ca0604deaabd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:07.799679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:07.814035 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e622fb46-0ed6-4b9d-8a7b-ca0604deaabd None None] MFA Rules not processed for user `36955d7d4505457696ab6177adb999a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:07.815317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e622fb46-0ed6-4b9d-8a7b-ca0604deaabd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:07.827569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1270/2542] 10.222.0.69 () {58 vars in 1018 bytes} [Wed Jul 20 20:08:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:08:07.838960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92319-88e1-474f-a1c1-49e874887b7c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:07.850492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af92319-88e1-474f-a1c1-49e874887b7c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:07.854258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2aabca-fe42-453e-a604-e8bf218f585a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:07.854895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2aabca-fe42-453e-a604-e8bf218f585a service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:07.855181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2aabca-fe42-453e-a604-e8bf218f585a service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:07.855460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2aabca-fe42-453e-a604-e8bf218f585a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:07.866239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2aabca-fe42-453e-a604-e8bf218f585a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:07.868429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2aabca-fe42-453e-a604-e8bf218f585a service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:07.879235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1273/2543] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:09.015958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622fb46-0ed6-4b9d-8a7b-ca0604deaabd None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:09.081267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e622fb46-0ed6-4b9d-8a7b-ca0604deaabd None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:09.086842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:09.087729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:09.088012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:09.088288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] PATH_INFO: `/v3/users/cc38506d39ff4fe0841f032820a170f0` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:09.089481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc38506d39ff4fe0841f032820a170f0)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:09.091486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:09.147285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc38506d39ff4fe0841f032820a170f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:09.160200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc38506d39ff4fe0841f032820a170f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:09.167397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc38506d39ff4fe0841f032820a170f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:09.168137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc38506d39ff4fe0841f032820a170f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:09.168741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc38506d39ff4fe0841f032820a170f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:09.182490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1271/2544] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:09 2022] DELETE /identity/v3/users/cc38506d39ff4fe0841f032820a170f0 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:09.186927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd3355-b4f2-4099-b87d-548124a13512 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:09.187449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd3355-b4f2-4099-b87d-548124a13512 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:09.187556 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dd3355-b4f2-4099-b87d-548124a13512 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:09.206867 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-93dd3355-b4f2-4099-b87d-548124a13512 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:09.208161 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-93dd3355-b4f2-4099-b87d-548124a13512 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:09.275165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1274/2545] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:08:09.284418 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:09.285841 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:09.348951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0584f1c-8d7b-4725-94ca-303375b72c09 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:09.353019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25251f-2e91-448d-bc56-6b0d9f691234 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:09.353702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25251f-2e91-448d-bc56-6b0d9f691234 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:09.353984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25251f-2e91-448d-bc56-6b0d9f691234 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:09.354258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb25251f-2e91-448d-bc56-6b0d9f691234 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:09.363308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25251f-2e91-448d-bc56-6b0d9f691234 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:09.364723 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb25251f-2e91-448d-bc56-6b0d9f691234 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:09.374855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1272/2546] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:09.512729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcbbf3-2d9d-4312-a536-cd9a2d366c9d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:09.513019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcbbf3-2d9d-4312-a536-cd9a2d366c9d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:09.513303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdcbbf3-2d9d-4312-a536-cd9a2d366c9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:09.532371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bbdcbbf3-2d9d-4312-a536-cd9a2d366c9d None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:09.533655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bbdcbbf3-2d9d-4312-a536-cd9a2d366c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:09.597836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1275/2547] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 20:08:09 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:09.610730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25251f-2e91-448d-bc56-6b0d9f691234 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:09.612128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cb25251f-2e91-448d-bc56-6b0d9f691234 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:09.635081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:09.687705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb25251f-2e91-448d-bc56-6b0d9f691234 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:09.693169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22301aa-8e8f-4d52-9cb5-3609fc7d2a8e service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:09.693855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22301aa-8e8f-4d52-9cb5-3609fc7d2a8e service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:09.694138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22301aa-8e8f-4d52-9cb5-3609fc7d2a8e service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:09.694412 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22301aa-8e8f-4d52-9cb5-3609fc7d2a8e service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:09.702929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22301aa-8e8f-4d52-9cb5-3609fc7d2a8e service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:09.704510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22301aa-8e8f-4d52-9cb5-3609fc7d2a8e service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:09.713195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1273/2548] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 20:08:09 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 20 20:08:10.153865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcbbf3-2d9d-4312-a536-cd9a2d366c9d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:10.155964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bbdcbbf3-2d9d-4312-a536-cd9a2d366c9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:10.182670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdcbbf3-2d9d-4312-a536-cd9a2d366c9d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:10.197347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:10.198264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:10.198560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:10.198833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] PATH_INFO: `/v3/projects/41a9a438a6954d91b926467901f02f7c` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:10.200035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] RBAC: Authorizing `identity:delete_project(project_id=41a9a438a6954d91b926467901f02f7c)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:10.201985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:10.262808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41a9a438a6954d91b926467901f02f7c', 'request_id': 'req-b8665863-c55e-4816-904f-038acff5114b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.272557 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41a9a438a6954d91b926467901f02f7c', 'request_id': 'req-b8665863-c55e-4816-904f-038acff5114b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.279974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41a9a438a6954d91b926467901f02f7c', 'request_id': 'req-b8665863-c55e-4816-904f-038acff5114b'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.292756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] The token cache is being invalidate because project 41a9a438a6954d91b926467901f02f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:10.293061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.293835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.295373 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1276/2549] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:10 2022] DELETE /identity/v3/projects/41a9a438a6954d91b926467901f02f7c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:10.299483 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22301aa-8e8f-4d52-9cb5-3609fc7d2a8e service swift] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:10.310768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22301aa-8e8f-4d52-9cb5-3609fc7d2a8e service swift] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:10.387080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:10.387968 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:10.388248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:10.388521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] PATH_INFO: `/v3/users/c9db7dc1a6744481b00dd54a9de69f75` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:10.389761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9db7dc1a6744481b00dd54a9de69f75)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:10.391702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:10.442827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9db7dc1a6744481b00dd54a9de69f75'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.463793 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9db7dc1a6744481b00dd54a9de69f75'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.471431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9db7dc1a6744481b00dd54a9de69f75'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.472328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9db7dc1a6744481b00dd54a9de69f75'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.472940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-57632448-9d19-4042-83e5-6b59207808d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9db7dc1a6744481b00dd54a9de69f75'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.487360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1274/2550] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:10 2022] DELETE /identity/v3/users/c9db7dc1a6744481b00dd54a9de69f75 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:10.793299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:10.794819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:10.802980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:10.808905 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8665863-c55e-4816-904f-038acff5114b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:10.869853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:10.870626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:10.870902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:10.871170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] PATH_INFO: `/v3/projects/b782b4506e4942aba391a3bc2379820d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:10.872361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] RBAC: Authorizing `identity:delete_project(project_id=b782b4506e4942aba391a3bc2379820d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:10.874295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:10.923472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b782b4506e4942aba391a3bc2379820d', 'request_id': 'req-d2b44f00-7e95-46d3-8e78-826c33ac809c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.930269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b782b4506e4942aba391a3bc2379820d', 'request_id': 'req-d2b44f00-7e95-46d3-8e78-826c33ac809c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.936019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b782b4506e4942aba391a3bc2379820d', 'request_id': 'req-d2b44f00-7e95-46d3-8e78-826c33ac809c'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.949741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] The token cache is being invalidate because project b782b4506e4942aba391a3bc2379820d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:10.950031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.950825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d2b44f00-7e95-46d3-8e78-826c33ac809c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:10.952566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1277/2551] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:10 2022] DELETE /identity/v3/projects/b782b4506e4942aba391a3bc2379820d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:10.957252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2aa3c5-4f57-463a-9f9d-5f95dad2a66e None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:10.957581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2aa3c5-4f57-463a-9f9d-5f95dad2a66e None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:10.957868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2aa3c5-4f57-463a-9f9d-5f95dad2a66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:10.975529 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0e2aa3c5-4f57-463a-9f9d-5f95dad2a66e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:10.977290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0e2aa3c5-4f57-463a-9f9d-5f95dad2a66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:11.010131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:11.040006 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1275/2552] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:11.048058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e31ee-8593-4f72-8323-b26693ede619 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.048352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e31ee-8593-4f72-8323-b26693ede619 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.048634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729e31ee-8593-4f72-8323-b26693ede619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.067341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-729e31ee-8593-4f72-8323-b26693ede619 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:11.068579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-729e31ee-8593-4f72-8323-b26693ede619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:11.076816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1278/2553] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:11.081096 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2aa3c5-4f57-463a-9f9d-5f95dad2a66e None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:11.091154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2aa3c5-4f57-463a-9f9d-5f95dad2a66e None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:11.096073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2089f-943b-403f-b9b3-b14473fde9a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:11.096727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2089f-943b-403f-b9b3-b14473fde9a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.097013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2089f-943b-403f-b9b3-b14473fde9a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.097289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e2089f-943b-403f-b9b3-b14473fde9a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.098076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2089f-943b-403f-b9b3-b14473fde9a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:08:11.098390 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2089f-943b-403f-b9b3-b14473fde9a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:11.100016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e2089f-943b-403f-b9b3-b14473fde9a0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:11.112143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1276/2554] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:08:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 20:08:11.116555 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e31ee-8593-4f72-8323-b26693ede619 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:11.126488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729e31ee-8593-4f72-8323-b26693ede619 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:11.131047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805613d-7857-4e85-b47d-2aa2b089233f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:11.131696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805613d-7857-4e85-b47d-2aa2b089233f admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.131984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805613d-7857-4e85-b47d-2aa2b089233f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.132291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8805613d-7857-4e85-b47d-2aa2b089233f admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.133013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805613d-7857-4e85-b47d-2aa2b089233f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:11.134973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8805613d-7857-4e85-b47d-2aa2b089233f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:11.159774 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1279/2555] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:11 2022] POST /identity/v3/projects => generated 361 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:11.164413 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2089f-943b-403f-b9b3-b14473fde9a0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:11.174603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e2089f-943b-403f-b9b3-b14473fde9a0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:11.229690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f4ed6-57ab-42f5-aa40-10109f57ad95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:11.230358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f4ed6-57ab-42f5-aa40-10109f57ad95 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.230642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f4ed6-57ab-42f5-aa40-10109f57ad95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.230919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad2f4ed6-57ab-42f5-aa40-10109f57ad95 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.231610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f4ed6-57ab-42f5-aa40-10109f57ad95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:11.233555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad2f4ed6-57ab-42f5-aa40-10109f57ad95 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:11.253219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1277/2556] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:11 2022] POST /identity/v3/users => generated 281 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:08:11.259215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805613d-7857-4e85-b47d-2aa2b089233f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:11.269890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8805613d-7857-4e85-b47d-2aa2b089233f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:11.274712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222dfd5-dc5d-46ff-916e-9f3a4a94676c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:11.275358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222dfd5-dc5d-46ff-916e-9f3a4a94676c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.275638 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222dfd5-dc5d-46ff-916e-9f3a4a94676c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.275910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6222dfd5-dc5d-46ff-916e-9f3a4a94676c admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.276622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222dfd5-dc5d-46ff-916e-9f3a4a94676c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:11.278233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6222dfd5-dc5d-46ff-916e-9f3a4a94676c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:11.289203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1280/2557] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:11.293732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f4ed6-57ab-42f5-aa40-10109f57ad95 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:11.303730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad2f4ed6-57ab-42f5-aa40-10109f57ad95 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:11.309067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bdadbe5-31a3-46e7-ae1f-b44578fa7139 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:11.309774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdadbe5-31a3-46e7-ae1f-b44578fa7139 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.310059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdadbe5-31a3-46e7-ae1f-b44578fa7139 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.310336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bdadbe5-31a3-46e7-ae1f-b44578fa7139 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.311066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdadbe5-31a3-46e7-ae1f-b44578fa7139 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:11.312666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bdadbe5-31a3-46e7-ae1f-b44578fa7139 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:11.323213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1278/2558] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:11.327293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222dfd5-dc5d-46ff-916e-9f3a4a94676c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:11.338092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6222dfd5-dc5d-46ff-916e-9f3a4a94676c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:11.342781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773311c-6df8-44e2-bbf4-fa8d2f88ddbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:11.343650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773311c-6df8-44e2-bbf4-fa8d2f88ddbc admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.343934 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773311c-6df8-44e2-bbf4-fa8d2f88ddbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.344209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c773311c-6df8-44e2-bbf4-fa8d2f88ddbc admin admin] PATH_INFO: `/v3/projects/0f7b14910ad94d7ab9e074c2f98c76ff/users/ee2590aec3084e95b48b2a7f4060d166/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.359850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773311c-6df8-44e2-bbf4-fa8d2f88ddbc admin admin] RBAC: Authorizing `identity:create_grant(project_id=0f7b14910ad94d7ab9e074c2f98c76ff, user_id=ee2590aec3084e95b48b2a7f4060d166, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:11.362544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c773311c-6df8-44e2-bbf4-fa8d2f88ddbc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:11.375391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1281/2559] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:11 2022] PUT /identity/v3/projects/0f7b14910ad94d7ab9e074c2f98c76ff/users/ee2590aec3084e95b48b2a7f4060d166/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:11.387159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a968bd-622c-4b4b-ae8a-14b0f12ca67b None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.387382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a968bd-622c-4b4b-ae8a-14b0f12ca67b None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.387660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a968bd-622c-4b4b-ae8a-14b0f12ca67b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.415360 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e2a968bd-622c-4b4b-ae8a-14b0f12ca67b None None] MFA Rules not processed for user `ee2590aec3084e95b48b2a7f4060d166`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:11.416573 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e2a968bd-622c-4b4b-ae8a-14b0f12ca67b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:11.481020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1279/2560] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:11.517682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773311c-6df8-44e2-bbf4-fa8d2f88ddbc admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:11.519390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c773311c-6df8-44e2-bbf4-fa8d2f88ddbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:11.551994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:11.578888 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c773311c-6df8-44e2-bbf4-fa8d2f88ddbc admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:11.582586 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d588461a-7926-4210-9ba5-464431a022c6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:11.583226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588461a-7926-4210-9ba5-464431a022c6 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.583513 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588461a-7926-4210-9ba5-464431a022c6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.583792 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d588461a-7926-4210-9ba5-464431a022c6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.592745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588461a-7926-4210-9ba5-464431a022c6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:11.594232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d588461a-7926-4210-9ba5-464431a022c6 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:11.603416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1282/2561] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:11 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:11.618176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a968bd-622c-4b4b-ae8a-14b0f12ca67b None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:11.645230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:11.688583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a968bd-622c-4b4b-ae8a-14b0f12ca67b None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:11.693139 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:11.694039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.694320 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.694593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] PATH_INFO: `/v3/users/ee2590aec3084e95b48b2a7f4060d166` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.696473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee2590aec3084e95b48b2a7f4060d166)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:11.698485 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:11.745484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee2590aec3084e95b48b2a7f4060d166'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:11.757342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee2590aec3084e95b48b2a7f4060d166'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:11.764519 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee2590aec3084e95b48b2a7f4060d166'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:11.765215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee2590aec3084e95b48b2a7f4060d166'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:11.765843 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee2590aec3084e95b48b2a7f4060d166'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:11.779117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1280/2562] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:11 2022] DELETE /identity/v3/users/ee2590aec3084e95b48b2a7f4060d166 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:11.783667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105c921-2e0a-4254-b798-4e85c14f9ee8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.783980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105c921-2e0a-4254-b798-4e85c14f9ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.784264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a105c921-2e0a-4254-b798-4e85c14f9ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.802280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a105c921-2e0a-4254-b798-4e85c14f9ee8 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:11.803553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a105c921-2e0a-4254-b798-4e85c14f9ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:11.865844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1283/2563] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:11.876292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:11.877998 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:11.939235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd0e257-79da-4d20-b798-992287ff89ba admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:11.947565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f26118-bf33-460b-954e-91cb7e7b298b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:11.948240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f26118-bf33-460b-954e-91cb7e7b298b service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:11.948526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f26118-bf33-460b-954e-91cb7e7b298b service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:11.948806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f26118-bf33-460b-954e-91cb7e7b298b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:11.959424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f26118-bf33-460b-954e-91cb7e7b298b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:11.961041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f26118-bf33-460b-954e-91cb7e7b298b service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:11.974403 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1281/2564] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:12.322917 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105c921-2e0a-4254-b798-4e85c14f9ee8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:12.324353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a105c921-2e0a-4254-b798-4e85c14f9ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:12.338239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a105c921-2e0a-4254-b798-4e85c14f9ee8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:12.344567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:12.345699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:12.345992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:12.346266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] PATH_INFO: `/v3/projects/0f7b14910ad94d7ab9e074c2f98c76ff` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:12.347554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] RBAC: Authorizing `identity:delete_project(project_id=0f7b14910ad94d7ab9e074c2f98c76ff)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:12.350175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:12.409102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0f7b14910ad94d7ab9e074c2f98c76ff', 'request_id': 'req-060f51ff-5039-48fc-8833-e1086adccfda'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:12.417523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0f7b14910ad94d7ab9e074c2f98c76ff', 'request_id': 'req-060f51ff-5039-48fc-8833-e1086adccfda'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:12.423390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0f7b14910ad94d7ab9e074c2f98c76ff', 'request_id': 'req-060f51ff-5039-48fc-8833-e1086adccfda'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:12.436162 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] The token cache is being invalidate because project 0f7b14910ad94d7ab9e074c2f98c76ff was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:12.436451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:12.437077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-060f51ff-5039-48fc-8833-e1086adccfda admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:12.438680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1284/2565] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:12 2022] DELETE /identity/v3/projects/0f7b14910ad94d7ab9e074c2f98c76ff => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:12.443598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da748c-eac4-4714-bb34-05bf4cac8763 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:12.444063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da748c-eac4-4714-bb34-05bf4cac8763 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:12.444167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0da748c-eac4-4714-bb34-05bf4cac8763 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:12.464152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f0da748c-eac4-4714-bb34-05bf4cac8763 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:12.465392 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f0da748c-eac4-4714-bb34-05bf4cac8763 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:12.511883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:12.531637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1282/2566] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:08:12.539100 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2b953-b449-4b6b-9934-c844c723433e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:12.539597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2b953-b449-4b6b-9934-c844c723433e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:12.539830 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e2b953-b449-4b6b-9934-c844c723433e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:12.558757 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a1e2b953-b449-4b6b-9934-c844c723433e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:12.559973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a1e2b953-b449-4b6b-9934-c844c723433e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:12.568051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1285/2567] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:12.573385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da748c-eac4-4714-bb34-05bf4cac8763 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:12.584220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0da748c-eac4-4714-bb34-05bf4cac8763 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:12.589858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc693f1-b6b7-4228-9249-4607565a680e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:12.590553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc693f1-b6b7-4228-9249-4607565a680e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:12.590822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc693f1-b6b7-4228-9249-4607565a680e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:12.591099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc693f1-b6b7-4228-9249-4607565a680e admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:12.591829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc693f1-b6b7-4228-9249-4607565a680e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:08:12.592133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc693f1-b6b7-4228-9249-4607565a680e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:12.593812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dc693f1-b6b7-4228-9249-4607565a680e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:12.610322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1283/2568] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:08:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:08:12.611152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2b953-b449-4b6b-9934-c844c723433e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:12.621400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e2b953-b449-4b6b-9934-c844c723433e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:12.626467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6b48b-39cf-4fee-92a0-ac116e209933 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:12.627131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6b48b-39cf-4fee-92a0-ac116e209933 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:12.627413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6b48b-39cf-4fee-92a0-ac116e209933 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:12.627686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d6b48b-39cf-4fee-92a0-ac116e209933 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:12.628379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6b48b-39cf-4fee-92a0-ac116e209933 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:12.630346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d6b48b-39cf-4fee-92a0-ac116e209933 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:12.671374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc693f1-b6b7-4228-9249-4607565a680e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:12.672397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1286/2569] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:12 2022] POST /identity/v3/projects => generated 375 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:12.683201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dc693f1-b6b7-4228-9249-4607565a680e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:12.754076 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2532f-08ac-4d1c-98fe-d954051f8d52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:12.754764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2532f-08ac-4d1c-98fe-d954051f8d52 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:12.755048 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2532f-08ac-4d1c-98fe-d954051f8d52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:12.755396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b2532f-08ac-4d1c-98fe-d954051f8d52 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:12.756080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2532f-08ac-4d1c-98fe-d954051f8d52 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:12.758598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b2532f-08ac-4d1c-98fe-d954051f8d52 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:12.785820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1284/2570] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:12 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:08:12.790290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6b48b-39cf-4fee-92a0-ac116e209933 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:12.801347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d6b48b-39cf-4fee-92a0-ac116e209933 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:12.806910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e43fbd-85dd-4254-97b5-96140d9bdc6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:12.807674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e43fbd-85dd-4254-97b5-96140d9bdc6e admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:12.808033 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e43fbd-85dd-4254-97b5-96140d9bdc6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:12.808383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e43fbd-85dd-4254-97b5-96140d9bdc6e admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:12.809126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e43fbd-85dd-4254-97b5-96140d9bdc6e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:12.810778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e43fbd-85dd-4254-97b5-96140d9bdc6e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:12.826316 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1287/2571] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:12.827514 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2532f-08ac-4d1c-98fe-d954051f8d52 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:12.839949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b2532f-08ac-4d1c-98fe-d954051f8d52 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:12.845349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9096879d-99ac-4365-94c4-4f58d80c9184 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:12.846345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096879d-99ac-4365-94c4-4f58d80c9184 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:12.846629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096879d-99ac-4365-94c4-4f58d80c9184 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:12.846905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9096879d-99ac-4365-94c4-4f58d80c9184 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:12.847635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096879d-99ac-4365-94c4-4f58d80c9184 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:12.849245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9096879d-99ac-4365-94c4-4f58d80c9184 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:12.864566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1285/2572] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:08:12.866183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e43fbd-85dd-4254-97b5-96140d9bdc6e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:12.877332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e43fbd-85dd-4254-97b5-96140d9bdc6e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:12.883534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219f8bd-91ce-4b26-b417-be30c968da8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:12.884751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219f8bd-91ce-4b26-b417-be30c968da8d admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:12.884751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219f8bd-91ce-4b26-b417-be30c968da8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:12.886153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a219f8bd-91ce-4b26-b417-be30c968da8d admin admin] PATH_INFO: `/v3/projects/53fafadad47f429f84a7f8334a558455/users/29ed520079d140a5b2ec759d833d2d74/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:12.900746 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:12.904350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219f8bd-91ce-4b26-b417-be30c968da8d admin admin] RBAC: Authorizing `identity:create_grant(project_id=53fafadad47f429f84a7f8334a558455, user_id=29ed520079d140a5b2ec759d833d2d74, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:12.907362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a219f8bd-91ce-4b26-b417-be30c968da8d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:12.921486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1288/2573] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:12 2022] PUT /identity/v3/projects/53fafadad47f429f84a7f8334a558455/users/29ed520079d140a5b2ec759d833d2d74/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:12.932300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98543fcd-7e4a-42a9-808c-d58d34c6407c None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:12.932724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98543fcd-7e4a-42a9-808c-d58d34c6407c None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:12.933074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98543fcd-7e4a-42a9-808c-d58d34c6407c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:12.963400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-98543fcd-7e4a-42a9-808c-d58d34c6407c None None] MFA Rules not processed for user `29ed520079d140a5b2ec759d833d2d74`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:12.964796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-98543fcd-7e4a-42a9-808c-d58d34c6407c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:13.032618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1286/2574] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:13.037350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219f8bd-91ce-4b26-b417-be30c968da8d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:13.102901 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a219f8bd-91ce-4b26-b417-be30c968da8d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:13.107706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b795080-e0dc-4605-be50-8f1a067acedf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:13.108390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b795080-e0dc-4605-be50-8f1a067acedf admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:13.108667 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b795080-e0dc-4605-be50-8f1a067acedf admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:13.108948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b795080-e0dc-4605-be50-8f1a067acedf admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:13.109726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b795080-e0dc-4605-be50-8f1a067acedf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:13.111655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b795080-e0dc-4605-be50-8f1a067acedf admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:13.140165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1289/2575] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:13 2022] POST /identity/v3/projects => generated 375 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:13.144835 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98543fcd-7e4a-42a9-808c-d58d34c6407c None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:13.155983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98543fcd-7e4a-42a9-808c-d58d34c6407c None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:13.197933 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:13.215658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69872b64-7f4c-4e2e-911c-85cfb151f193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:13.216348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69872b64-7f4c-4e2e-911c-85cfb151f193 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:13.216637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69872b64-7f4c-4e2e-911c-85cfb151f193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:13.216928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69872b64-7f4c-4e2e-911c-85cfb151f193 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:13.217645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69872b64-7f4c-4e2e-911c-85cfb151f193 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:13.219614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69872b64-7f4c-4e2e-911c-85cfb151f193 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:13.244853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1287/2576] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:13 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:08:13.246158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b795080-e0dc-4605-be50-8f1a067acedf admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:13.256881 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b795080-e0dc-4605-be50-8f1a067acedf admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:13.261735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce9001-40c6-43e7-8b64-83559c36ad88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:13.262415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce9001-40c6-43e7-8b64-83559c36ad88 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:13.262701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce9001-40c6-43e7-8b64-83559c36ad88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:13.262982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cce9001-40c6-43e7-8b64-83559c36ad88 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:13.263707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce9001-40c6-43e7-8b64-83559c36ad88 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:13.265662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cce9001-40c6-43e7-8b64-83559c36ad88 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:13.287938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1290/2577] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:08:13.289826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69872b64-7f4c-4e2e-911c-85cfb151f193 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:13.302783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69872b64-7f4c-4e2e-911c-85cfb151f193 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:13.308828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23935744-69e5-423a-b39a-01e60ee9a524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:13.309847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23935744-69e5-423a-b39a-01e60ee9a524 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:13.310181 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23935744-69e5-423a-b39a-01e60ee9a524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:13.310524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23935744-69e5-423a-b39a-01e60ee9a524 admin admin] PATH_INFO: `/v3/projects/7873208bfa284e0091ad7fe0c78dd59b/users/b9f98bb84de04133be8394e65d074138/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:13.335946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23935744-69e5-423a-b39a-01e60ee9a524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7873208bfa284e0091ad7fe0c78dd59b, user_id=b9f98bb84de04133be8394e65d074138, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:13.339133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23935744-69e5-423a-b39a-01e60ee9a524 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:13.358240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1288/2578] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:13 2022] PUT /identity/v3/projects/7873208bfa284e0091ad7fe0c78dd59b/users/b9f98bb84de04133be8394e65d074138/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:13.363649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce9001-40c6-43e7-8b64-83559c36ad88 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:13.383105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cce9001-40c6-43e7-8b64-83559c36ad88 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:13.460156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73330d28-060f-4db6-acb1-ef5977d98c96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:13.460896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73330d28-060f-4db6-acb1-ef5977d98c96 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:13.461187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73330d28-060f-4db6-acb1-ef5977d98c96 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:13.462106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73330d28-060f-4db6-acb1-ef5977d98c96 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:13.462947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73330d28-060f-4db6-acb1-ef5977d98c96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:13.464588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73330d28-060f-4db6-acb1-ef5977d98c96 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:13.481547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1291/2579] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:13 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:13.483611 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23935744-69e5-423a-b39a-01e60ee9a524 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:13.495469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23935744-69e5-423a-b39a-01e60ee9a524 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:13.500460 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b3374-6da7-4998-8fde-3e73526eca2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:13.501363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b3374-6da7-4998-8fde-3e73526eca2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:13.501684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b3374-6da7-4998-8fde-3e73526eca2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:13.501964 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff1b3374-6da7-4998-8fde-3e73526eca2a admin admin] PATH_INFO: `/v3/projects/7873208bfa284e0091ad7fe0c78dd59b/users/b9f98bb84de04133be8394e65d074138/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:13.505243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b3374-6da7-4998-8fde-3e73526eca2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7873208bfa284e0091ad7fe0c78dd59b, user_id=b9f98bb84de04133be8394e65d074138, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:13.508183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff1b3374-6da7-4998-8fde-3e73526eca2a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:13.519134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1289/2580] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:13 2022] PUT /identity/v3/projects/7873208bfa284e0091ad7fe0c78dd59b/users/b9f98bb84de04133be8394e65d074138/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:13.530487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967f7d-197a-4a0b-9fe2-f495777c9d1d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:13.530788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967f7d-197a-4a0b-9fe2-f495777c9d1d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:13.531068 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c967f7d-197a-4a0b-9fe2-f495777c9d1d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:13.558532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0c967f7d-197a-4a0b-9fe2-f495777c9d1d None None] MFA Rules not processed for user `b9f98bb84de04133be8394e65d074138`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:13.559690 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0c967f7d-197a-4a0b-9fe2-f495777c9d1d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:13.609612 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:13.633293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1292/2581] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:13 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:13.646790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b3374-6da7-4998-8fde-3e73526eca2a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:13.649117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ff1b3374-6da7-4998-8fde-3e73526eca2a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:13.718889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff1b3374-6da7-4998-8fde-3e73526eca2a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:13.723095 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-825c5b67-3de6-479f-b627-7bfc6264e4b6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:13.723767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c5b67-3de6-479f-b627-7bfc6264e4b6 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:13.724053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c5b67-3de6-479f-b627-7bfc6264e4b6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:13.724339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825c5b67-3de6-479f-b627-7bfc6264e4b6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:13.784126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c5b67-3de6-479f-b627-7bfc6264e4b6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:13.785619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-825c5b67-3de6-479f-b627-7bfc6264e4b6 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:13.799459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1290/2582] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:16.399709 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c967f7d-197a-4a0b-9fe2-f495777c9d1d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:16.467230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c967f7d-197a-4a0b-9fe2-f495777c9d1d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:16.472168 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:16.473043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:16.473322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:16.473629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] PATH_INFO: `/v3/users/29ed520079d140a5b2ec759d833d2d74` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:16.474833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] RBAC: Authorizing `identity:delete_user(user_id=29ed520079d140a5b2ec759d833d2d74)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:16.477342 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:16.528124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29ed520079d140a5b2ec759d833d2d74'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:16.541951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29ed520079d140a5b2ec759d833d2d74'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:16.549251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29ed520079d140a5b2ec759d833d2d74'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:16.550135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29ed520079d140a5b2ec759d833d2d74'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:16.550767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29ed520079d140a5b2ec759d833d2d74'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:16.574333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1293/2583] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:16 2022] DELETE /identity/v3/users/29ed520079d140a5b2ec759d833d2d74 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:16.578978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e294-8572-4116-8045-62c114dd2955 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:16.579524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e294-8572-4116-8045-62c114dd2955 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:16.579524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff6e294-8572-4116-8045-62c114dd2955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:16.598097 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7ff6e294-8572-4116-8045-62c114dd2955 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:16.599322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7ff6e294-8572-4116-8045-62c114dd2955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:16.647760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:16.671158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1291/2584] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:16.681687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:16.683446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:16.746838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d9439-b985-4e42-b446-412f0b440637 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:16.758334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a8b8f-5c96-4b51-a89d-b98123871d29 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:16.758334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a8b8f-5c96-4b51-a89d-b98123871d29 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:16.758334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a8b8f-5c96-4b51-a89d-b98123871d29 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:16.758334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180a8b8f-5c96-4b51-a89d-b98123871d29 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:16.768932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a8b8f-5c96-4b51-a89d-b98123871d29 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:16.770674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180a8b8f-5c96-4b51-a89d-b98123871d29 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:16.782980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1294/2585] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:08:17.131003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6e294-8572-4116-8045-62c114dd2955 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:17.133349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7ff6e294-8572-4116-8045-62c114dd2955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:17.147520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff6e294-8572-4116-8045-62c114dd2955 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:17.171275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:17.172205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:17.173810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:17.174080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] PATH_INFO: `/v3/projects/53fafadad47f429f84a7f8334a558455` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:17.175436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=53fafadad47f429f84a7f8334a558455)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:17.177452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:17.251624 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53fafadad47f429f84a7f8334a558455', 'request_id': 'req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.259051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53fafadad47f429f84a7f8334a558455', 'request_id': 'req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.267098 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53fafadad47f429f84a7f8334a558455', 'request_id': 'req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.281538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] The token cache is being invalidate because project 53fafadad47f429f84a7f8334a558455 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:17.281913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.282599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.284176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1292/2586] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:17 2022] DELETE /identity/v3/projects/53fafadad47f429f84a7f8334a558455 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:17.288350 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a8b8f-5c96-4b51-a89d-b98123871d29 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:17.299483 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180a8b8f-5c96-4b51-a89d-b98123871d29 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:17.305447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:17.369323 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:17.370233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:17.370515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:17.370793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] PATH_INFO: `/v3/users/b9f98bb84de04133be8394e65d074138` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:17.372014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b9f98bb84de04133be8394e65d074138)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:17.373981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:17.427940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b9f98bb84de04133be8394e65d074138'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.447715 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b9f98bb84de04133be8394e65d074138'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.456254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b9f98bb84de04133be8394e65d074138'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.457195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b9f98bb84de04133be8394e65d074138'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.457823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-11993b6a-dece-4de8-9741-b584eb9b40a5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b9f98bb84de04133be8394e65d074138'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.471778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1295/2587] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:17 2022] DELETE /identity/v3/users/b9f98bb84de04133be8394e65d074138 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:17.777180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:17.779838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:17.814081 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0e25f0-3819-41ad-a55e-b0e12ced65b0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:17.878983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:17.879873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:17.880201 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:17.880426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] PATH_INFO: `/v3/projects/7873208bfa284e0091ad7fe0c78dd59b` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:17.881671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7873208bfa284e0091ad7fe0c78dd59b)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:17.883807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:17.949877 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7873208bfa284e0091ad7fe0c78dd59b', 'request_id': 'req-db975f53-384d-4ce0-b0a3-f3a52afcb809'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.963525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7873208bfa284e0091ad7fe0c78dd59b', 'request_id': 'req-db975f53-384d-4ce0-b0a3-f3a52afcb809'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.974343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7873208bfa284e0091ad7fe0c78dd59b', 'request_id': 'req-db975f53-384d-4ce0-b0a3-f3a52afcb809'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.989419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] The token cache is being invalidate because project 7873208bfa284e0091ad7fe0c78dd59b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:17.989751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.990421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-db975f53-384d-4ce0-b0a3-f3a52afcb809 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:17.992662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1293/2588] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:17 2022] DELETE /identity/v3/projects/7873208bfa284e0091ad7fe0c78dd59b => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:17.997919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee1dc8-4d50-4df9-addc-c367ee05ae80 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:17.998220 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee1dc8-4d50-4df9-addc-c367ee05ae80 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:17.998501 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffee1dc8-4d50-4df9-addc-c367ee05ae80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:18.017318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ffee1dc8-4d50-4df9-addc-c367ee05ae80 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:18.018552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ffee1dc8-4d50-4df9-addc-c367ee05ae80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:18.109727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1296/2589] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:18.115066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f144bd-f022-45e3-ab50-bf7eaaa2e0cb None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:18.115356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f144bd-f022-45e3-ab50-bf7eaaa2e0cb None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:18.115690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f144bd-f022-45e3-ab50-bf7eaaa2e0cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:18.124170 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:18.143066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b6f144bd-f022-45e3-ab50-bf7eaaa2e0cb None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:18.146396 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b6f144bd-f022-45e3-ab50-bf7eaaa2e0cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:18.159679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1294/2590] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:18.160989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee1dc8-4d50-4df9-addc-c367ee05ae80 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:18.172927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffee1dc8-4d50-4df9-addc-c367ee05ae80 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:18.178540 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de5080-7cb6-410c-9d28-fca754ae5533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:18.179290 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de5080-7cb6-410c-9d28-fca754ae5533 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:18.179851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de5080-7cb6-410c-9d28-fca754ae5533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:18.180210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0de5080-7cb6-410c-9d28-fca754ae5533 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:18.181134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de5080-7cb6-410c-9d28-fca754ae5533 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:08:18.181590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de5080-7cb6-410c-9d28-fca754ae5533 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:18.183421 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0de5080-7cb6-410c-9d28-fca754ae5533 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:18.201359 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1297/2591] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:08:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 20:08:18.205314 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f144bd-f022-45e3-ab50-bf7eaaa2e0cb None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:18.215966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f144bd-f022-45e3-ab50-bf7eaaa2e0cb None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:18.221764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83479ae9-820c-408c-b667-a3eda9135bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:18.222520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83479ae9-820c-408c-b667-a3eda9135bb4 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:18.229593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83479ae9-820c-408c-b667-a3eda9135bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:18.230025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83479ae9-820c-408c-b667-a3eda9135bb4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:18.232474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83479ae9-820c-408c-b667-a3eda9135bb4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:18.234599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83479ae9-820c-408c-b667-a3eda9135bb4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:18.294627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1295/2592] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:18 2022] POST /identity/v3/projects => generated 375 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:18.298833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de5080-7cb6-410c-9d28-fca754ae5533 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:18.311115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0de5080-7cb6-410c-9d28-fca754ae5533 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:18.352623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:18.425660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abce428-ee13-4387-8dbc-198e053e97ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:18.426346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abce428-ee13-4387-8dbc-198e053e97ea admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:18.426636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abce428-ee13-4387-8dbc-198e053e97ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:18.426912 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abce428-ee13-4387-8dbc-198e053e97ea admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:18.427620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abce428-ee13-4387-8dbc-198e053e97ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:18.429743 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9abce428-ee13-4387-8dbc-198e053e97ea admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:18.453611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1298/2593] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:18 2022] POST /identity/v3/users => generated 288 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:08:18.456504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83479ae9-820c-408c-b667-a3eda9135bb4 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:18.467607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83479ae9-820c-408c-b667-a3eda9135bb4 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:18.472530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc183d-7642-4534-980a-08ed4ec1dfbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:18.473190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc183d-7642-4534-980a-08ed4ec1dfbc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:18.473475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc183d-7642-4534-980a-08ed4ec1dfbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:18.473783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fc183d-7642-4534-980a-08ed4ec1dfbc admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:18.474509 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc183d-7642-4534-980a-08ed4ec1dfbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:18.476100 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fc183d-7642-4534-980a-08ed4ec1dfbc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:18.486749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1296/2594] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:18.490967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abce428-ee13-4387-8dbc-198e053e97ea admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:18.501280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abce428-ee13-4387-8dbc-198e053e97ea admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:18.506047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729c92ab-8af3-4d82-ad7f-f823c9fd1931 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:18.506702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c92ab-8af3-4d82-ad7f-f823c9fd1931 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:18.506988 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c92ab-8af3-4d82-ad7f-f823c9fd1931 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:18.507269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729c92ab-8af3-4d82-ad7f-f823c9fd1931 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:18.508549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c92ab-8af3-4d82-ad7f-f823c9fd1931 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:18.510184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729c92ab-8af3-4d82-ad7f-f823c9fd1931 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:18.520523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1299/2595] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:18.525745 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc183d-7642-4534-980a-08ed4ec1dfbc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:18.536050 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fc183d-7642-4534-980a-08ed4ec1dfbc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:18.541131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d043f-593c-4926-9667-96abafd45a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:18.542066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d043f-593c-4926-9667-96abafd45a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:18.542352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d043f-593c-4926-9667-96abafd45a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:18.542668 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959d043f-593c-4926-9667-96abafd45a1c admin admin] PATH_INFO: `/v3/projects/549224f283684ef6848e8a4356303d40/users/3a13ad1f99ee4042b5c3d7ecef5e88eb/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:18.558715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d043f-593c-4926-9667-96abafd45a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=549224f283684ef6848e8a4356303d40, user_id=3a13ad1f99ee4042b5c3d7ecef5e88eb, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:18.561420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959d043f-593c-4926-9667-96abafd45a1c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:18.574921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1297/2596] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:18 2022] PUT /identity/v3/projects/549224f283684ef6848e8a4356303d40/users/3a13ad1f99ee4042b5c3d7ecef5e88eb/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:18.642794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af6203-1d30-44b6-a6d7-e902a1924c3d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:18.643087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af6203-1d30-44b6-a6d7-e902a1924c3d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:18.643365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af6203-1d30-44b6-a6d7-e902a1924c3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:18.681967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-58af6203-1d30-44b6-a6d7-e902a1924c3d None None] MFA Rules not processed for user `3a13ad1f99ee4042b5c3d7ecef5e88eb`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:18.681967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-58af6203-1d30-44b6-a6d7-e902a1924c3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:18.753237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1300/2597] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:18.762813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d043f-593c-4926-9667-96abafd45a1c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:18.764915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-959d043f-593c-4926-9667-96abafd45a1c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:18.832789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959d043f-593c-4926-9667-96abafd45a1c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:18.836793 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b67e5-b909-4299-af40-8c55e3a983f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:18.837484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b67e5-b909-4299-af40-8c55e3a983f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:18.837790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b67e5-b909-4299-af40-8c55e3a983f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:18.838065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-168b67e5-b909-4299-af40-8c55e3a983f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:18.847906 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b67e5-b909-4299-af40-8c55e3a983f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:18.849355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-168b67e5-b909-4299-af40-8c55e3a983f7 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:18.859883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1298/2598] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:18 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:28.745863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=59,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:28.854975 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=52,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:45.255914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c0f24-4f98-4bcc-b159-68356200b64a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:45.256461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c0f24-4f98-4bcc-b159-68356200b64a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:45.256616 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-230c0f24-4f98-4bcc-b159-68356200b64a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:45.279184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-230c0f24-4f98-4bcc-b159-68356200b64a None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:45.280454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-230c0f24-4f98-4bcc-b159-68356200b64a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:45.348135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1301/2599] 10.222.0.69 () {58 vars in 967 bytes} [Wed Jul 20 20:08:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:45.362874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b67e5-b909-4299-af40-8c55e3a983f7 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:45.364446 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-168b67e5-b909-4299-af40-8c55e3a983f7 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:45.450500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-168b67e5-b909-4299-af40-8c55e3a983f7 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:45.458228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-424b922e-6ced-407c-93bf-a4ca1efe8064 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:45.458974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b922e-6ced-407c-93bf-a4ca1efe8064 service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:45.459276 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b922e-6ced-407c-93bf-a4ca1efe8064 service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:45.459572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-424b922e-6ced-407c-93bf-a4ca1efe8064 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:45.488718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b922e-6ced-407c-93bf-a4ca1efe8064 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:45.488718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-424b922e-6ced-407c-93bf-a4ca1efe8064 service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:45.492302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1299/2600] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 20:08:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 20:08:47.831423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c0f24-4f98-4bcc-b159-68356200b64a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:47.893957 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-230c0f24-4f98-4bcc-b159-68356200b64a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:47.898409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:47.899270 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:47.899548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:47.899824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] PATH_INFO: `/v3/users/3a13ad1f99ee4042b5c3d7ecef5e88eb` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:47.901130 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=3a13ad1f99ee4042b5c3d7ecef5e88eb)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:47.903076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:47.948930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3a13ad1f99ee4042b5c3d7ecef5e88eb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:47.961534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3a13ad1f99ee4042b5c3d7ecef5e88eb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:47.968708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3a13ad1f99ee4042b5c3d7ecef5e88eb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:47.969390 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3a13ad1f99ee4042b5c3d7ecef5e88eb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:47.970106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3a13ad1f99ee4042b5c3d7ecef5e88eb'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:47.983547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1302/2601] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:47 2022] DELETE /identity/v3/users/3a13ad1f99ee4042b5c3d7ecef5e88eb => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:47.988207 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e917d9a-5eeb-40e5-b006-10bf0bcfe3da None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:47.988712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e917d9a-5eeb-40e5-b006-10bf0bcfe3da None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:47.988902 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e917d9a-5eeb-40e5-b006-10bf0bcfe3da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:48.006589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0e917d9a-5eeb-40e5-b006-10bf0bcfe3da None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:48.007796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0e917d9a-5eeb-40e5-b006-10bf0bcfe3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:48.066224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1300/2602] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:48.075967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:48.077367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:48.108008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:48.139533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb92b66f-4234-4093-9a3e-67515c0351ec admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:48.143216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bfda8a-b902-4f11-a497-93e73cfac678 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:48.143863 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfda8a-b902-4f11-a497-93e73cfac678 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:48.144142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfda8a-b902-4f11-a497-93e73cfac678 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:48.144416 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bfda8a-b902-4f11-a497-93e73cfac678 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:48.153914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfda8a-b902-4f11-a497-93e73cfac678 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:48.155333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39bfda8a-b902-4f11-a497-93e73cfac678 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:48.164626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1303/2603] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:48.472647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e917d9a-5eeb-40e5-b006-10bf0bcfe3da None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:48.474376 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0e917d9a-5eeb-40e5-b006-10bf0bcfe3da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:48.488198 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e917d9a-5eeb-40e5-b006-10bf0bcfe3da None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:48.493357 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:48.494597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:48.495715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:48.496045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] PATH_INFO: `/v3/projects/549224f283684ef6848e8a4356303d40` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:48.497492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=549224f283684ef6848e8a4356303d40)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:48.500403 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:48.553244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '549224f283684ef6848e8a4356303d40', 'request_id': 'req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:48.559449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '549224f283684ef6848e8a4356303d40', 'request_id': 'req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:48.568011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '549224f283684ef6848e8a4356303d40', 'request_id': 'req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:48.580257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] The token cache is being invalidate because project 549224f283684ef6848e8a4356303d40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:48.581694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:48.582436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-22f3e5ba-e1a1-48c1-a643-f6e46c5a45c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:48.584120 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1301/2604] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:48 2022] DELETE /identity/v3/projects/549224f283684ef6848e8a4356303d40 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:48.590052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7eade-b26b-415d-b2c5-1a43248e5034 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:48.590335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7eade-b26b-415d-b2c5-1a43248e5034 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:48.590610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee7eade-b26b-415d-b2c5-1a43248e5034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:48.610690 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-fee7eade-b26b-415d-b2c5-1a43248e5034 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:48.611909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fee7eade-b26b-415d-b2c5-1a43248e5034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:48.680362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1304/2605] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:48.688602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7da35f-55a4-4d9f-8565-9ca869dfe49f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:48.689063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7da35f-55a4-4d9f-8565-9ca869dfe49f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:48.689575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7da35f-55a4-4d9f-8565-9ca869dfe49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:48.716307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cc7da35f-55a4-4d9f-8565-9ca869dfe49f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:48.717536 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cc7da35f-55a4-4d9f-8565-9ca869dfe49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:48.729564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1302/2606] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:48.731048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7eade-b26b-415d-b2c5-1a43248e5034 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:48.746176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee7eade-b26b-415d-b2c5-1a43248e5034 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:48.751176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b4c2c-b3f7-4a8e-8abb-c7f169cc1240 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:48.752217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b4c2c-b3f7-4a8e-8abb-c7f169cc1240 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:48.752780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b4c2c-b3f7-4a8e-8abb-c7f169cc1240 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:48.753062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-137b4c2c-b3f7-4a8e-8abb-c7f169cc1240 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:48.753977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b4c2c-b3f7-4a8e-8abb-c7f169cc1240 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:08:48.754291 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b4c2c-b3f7-4a8e-8abb-c7f169cc1240 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:48.755897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-137b4c2c-b3f7-4a8e-8abb-c7f169cc1240 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:48.770987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1305/2607] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:08:48.775542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7da35f-55a4-4d9f-8565-9ca869dfe49f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:48.786182 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7da35f-55a4-4d9f-8565-9ca869dfe49f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:48.791038 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab1a24-16c7-4f5d-932b-94be72c4dba5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:48.791737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab1a24-16c7-4f5d-932b-94be72c4dba5 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:48.792022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab1a24-16c7-4f5d-932b-94be72c4dba5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:48.792546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ab1a24-16c7-4f5d-932b-94be72c4dba5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:48.793270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab1a24-16c7-4f5d-932b-94be72c4dba5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:48.795242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ab1a24-16c7-4f5d-932b-94be72c4dba5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:48.799160 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:48.829828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1303/2608] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:48 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:08:48.834983 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b4c2c-b3f7-4a8e-8abb-c7f169cc1240 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:48.850204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-137b4c2c-b3f7-4a8e-8abb-c7f169cc1240 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:48.917445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a59c05-d57d-47a5-aa4e-eb0e895253ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:48.919039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a59c05-d57d-47a5-aa4e-eb0e895253ba admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:48.919328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a59c05-d57d-47a5-aa4e-eb0e895253ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:48.919615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a59c05-d57d-47a5-aa4e-eb0e895253ba admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:48.920347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a59c05-d57d-47a5-aa4e-eb0e895253ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:48.922319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4a59c05-d57d-47a5-aa4e-eb0e895253ba admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:48.945875 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1306/2609] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:48 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:08:48.950764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab1a24-16c7-4f5d-932b-94be72c4dba5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:48.961554 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ab1a24-16c7-4f5d-932b-94be72c4dba5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:48.966959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c5c7d-d9ab-4b6a-8ec3-6955ef3738e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:48.967635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c5c7d-d9ab-4b6a-8ec3-6955ef3738e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:48.967921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c5c7d-d9ab-4b6a-8ec3-6955ef3738e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:48.968205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29c5c7d-d9ab-4b6a-8ec3-6955ef3738e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:48.969172 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c5c7d-d9ab-4b6a-8ec3-6955ef3738e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:48.971449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b29c5c7d-d9ab-4b6a-8ec3-6955ef3738e0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:48.983518 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1304/2610] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:48.987833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a59c05-d57d-47a5-aa4e-eb0e895253ba admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:48.998402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4a59c05-d57d-47a5-aa4e-eb0e895253ba admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.003151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ff9e3c5-fc38-4893-9c0d-1e6e53138ccd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.003814 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9e3c5-fc38-4893-9c0d-1e6e53138ccd admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.004100 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9e3c5-fc38-4893-9c0d-1e6e53138ccd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.004379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ff9e3c5-fc38-4893-9c0d-1e6e53138ccd admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.005115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9e3c5-fc38-4893-9c0d-1e6e53138ccd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.006753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ff9e3c5-fc38-4893-9c0d-1e6e53138ccd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.017312 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1307/2611] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:49.021630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c5c7d-d9ab-4b6a-8ec3-6955ef3738e0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.032256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b29c5c7d-d9ab-4b6a-8ec3-6955ef3738e0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.036911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab24f73-76c5-483d-9aff-d7868ac38409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.037829 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab24f73-76c5-483d-9aff-d7868ac38409 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.038122 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab24f73-76c5-483d-9aff-d7868ac38409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.038404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab24f73-76c5-483d-9aff-d7868ac38409 admin admin] PATH_INFO: `/v3/projects/61a09f54e29842bcb919c1cb18fd5e6b/users/1546f2768de34bf9855f5f6d80fb2671/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.053979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab24f73-76c5-483d-9aff-d7868ac38409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=61a09f54e29842bcb919c1cb18fd5e6b, user_id=1546f2768de34bf9855f5f6d80fb2671, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.056667 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab24f73-76c5-483d-9aff-d7868ac38409 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.070972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1305/2612] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:49 2022] PUT /identity/v3/projects/61a09f54e29842bcb919c1cb18fd5e6b/users/1546f2768de34bf9855f5f6d80fb2671/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:49.082056 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afad05c-f2e0-4503-a97f-29b18a5250e6 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.082283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afad05c-f2e0-4503-a97f-29b18a5250e6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.082587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afad05c-f2e0-4503-a97f-29b18a5250e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.108609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3afad05c-f2e0-4503-a97f-29b18a5250e6 None None] MFA Rules not processed for user `1546f2768de34bf9855f5f6d80fb2671`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:49.109950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3afad05c-f2e0-4503-a97f-29b18a5250e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:49.129797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:49.174694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1308/2613] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:49.179305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab24f73-76c5-483d-9aff-d7868ac38409 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.240761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab24f73-76c5-483d-9aff-d7868ac38409 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.246278 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd35c6-c036-4640-8062-eeff9ce80a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.246939 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd35c6-c036-4640-8062-eeff9ce80a85 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.247226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd35c6-c036-4640-8062-eeff9ce80a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.247505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33fd35c6-c036-4640-8062-eeff9ce80a85 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.248188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd35c6-c036-4640-8062-eeff9ce80a85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.250158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33fd35c6-c036-4640-8062-eeff9ce80a85 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.274528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1306/2614] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:49 2022] POST /identity/v3/projects => generated 383 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:49.278703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afad05c-f2e0-4503-a97f-29b18a5250e6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.289808 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3afad05c-f2e0-4503-a97f-29b18a5250e6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.348248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b97d-c08d-429e-af7f-05377ae347f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.349117 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b97d-c08d-429e-af7f-05377ae347f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.349404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b97d-c08d-429e-af7f-05377ae347f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.349711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a45b97d-c08d-429e-af7f-05377ae347f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.350413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b97d-c08d-429e-af7f-05377ae347f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.352360 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a45b97d-c08d-429e-af7f-05377ae347f0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.373479 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1309/2615] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:49 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:49.377765 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd35c6-c036-4640-8062-eeff9ce80a85 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.388322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33fd35c6-c036-4640-8062-eeff9ce80a85 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.392931 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49cc7a-909c-4d1a-a492-9dcc083b28d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.393638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49cc7a-909c-4d1a-a492-9dcc083b28d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.393924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49cc7a-909c-4d1a-a492-9dcc083b28d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.394202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb49cc7a-909c-4d1a-a492-9dcc083b28d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.394930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49cc7a-909c-4d1a-a492-9dcc083b28d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.396529 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb49cc7a-909c-4d1a-a492-9dcc083b28d2 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.407145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1307/2616] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:49.411137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b97d-c08d-429e-af7f-05377ae347f0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.421838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a45b97d-c08d-429e-af7f-05377ae347f0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.426482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc56248d-6bca-4a70-91a6-4105b1689ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.427143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56248d-6bca-4a70-91a6-4105b1689ddf admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.427430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56248d-6bca-4a70-91a6-4105b1689ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.427709 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc56248d-6bca-4a70-91a6-4105b1689ddf admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.428441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56248d-6bca-4a70-91a6-4105b1689ddf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.430080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc56248d-6bca-4a70-91a6-4105b1689ddf admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.440615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1310/2617] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:49.444832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49cc7a-909c-4d1a-a492-9dcc083b28d2 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.455143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb49cc7a-909c-4d1a-a492-9dcc083b28d2 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.459697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e7468-9d4e-4331-88e0-3f3e3102350b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.460574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e7468-9d4e-4331-88e0-3f3e3102350b admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.460858 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e7468-9d4e-4331-88e0-3f3e3102350b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.461135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-734e7468-9d4e-4331-88e0-3f3e3102350b admin admin] PATH_INFO: `/v3/projects/95288af0d57e410fb434282f4cfdefa1/users/aa073a1a2e4944f1b74053c8dd109fe0/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.477114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e7468-9d4e-4331-88e0-3f3e3102350b admin admin] RBAC: Authorizing `identity:create_grant(project_id=95288af0d57e410fb434282f4cfdefa1, user_id=aa073a1a2e4944f1b74053c8dd109fe0, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.480455 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-734e7468-9d4e-4331-88e0-3f3e3102350b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.492887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1308/2618] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:49 2022] PUT /identity/v3/projects/95288af0d57e410fb434282f4cfdefa1/users/aa073a1a2e4944f1b74053c8dd109fe0/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:49.504648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e76f0-6d43-4399-9191-8a2d4b8abd56 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.504943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e76f0-6d43-4399-9191-8a2d4b8abd56 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.505228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3e76f0-6d43-4399-9191-8a2d4b8abd56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.532235 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-6b3e76f0-6d43-4399-9191-8a2d4b8abd56 None None] MFA Rules not processed for user `aa073a1a2e4944f1b74053c8dd109fe0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:49.533461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-6b3e76f0-6d43-4399-9191-8a2d4b8abd56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:49.599544 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1311/2619] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:49 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:49.603974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e7468-9d4e-4331-88e0-3f3e3102350b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.665086 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-734e7468-9d4e-4331-88e0-3f3e3102350b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.670532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4b0d9-2b06-4d75-890d-dabcf2fed2e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.671195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4b0d9-2b06-4d75-890d-dabcf2fed2e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.671478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4b0d9-2b06-4d75-890d-dabcf2fed2e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.671773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db4b0d9-2b06-4d75-890d-dabcf2fed2e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.672450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4b0d9-2b06-4d75-890d-dabcf2fed2e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.674399 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db4b0d9-2b06-4d75-890d-dabcf2fed2e9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.677581 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:49.699963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1309/2620] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:49 2022] POST /identity/v3/projects => generated 385 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:08:49.724866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e76f0-6d43-4399-9191-8a2d4b8abd56 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.724866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3e76f0-6d43-4399-9191-8a2d4b8abd56 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.752437 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:49.780961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30e46f-24f5-4a53-8dc9-bd98105fe3fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.781742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e46f-24f5-4a53-8dc9-bd98105fe3fa admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.782048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e46f-24f5-4a53-8dc9-bd98105fe3fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.782328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e30e46f-24f5-4a53-8dc9-bd98105fe3fa admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.783054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30e46f-24f5-4a53-8dc9-bd98105fe3fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.784985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e30e46f-24f5-4a53-8dc9-bd98105fe3fa admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.804960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1312/2621] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:49 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:49.809231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4b0d9-2b06-4d75-890d-dabcf2fed2e9 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.819474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db4b0d9-2b06-4d75-890d-dabcf2fed2e9 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.824311 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a78937-2e99-4361-9996-73723627ea8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.824972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a78937-2e99-4361-9996-73723627ea8d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.825259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a78937-2e99-4361-9996-73723627ea8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.825571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a78937-2e99-4361-9996-73723627ea8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.826304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a78937-2e99-4361-9996-73723627ea8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.827937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a78937-2e99-4361-9996-73723627ea8d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.838400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1310/2622] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:49.843162 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30e46f-24f5-4a53-8dc9-bd98105fe3fa admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.853499 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e30e46f-24f5-4a53-8dc9-bd98105fe3fa admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.858967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6267f-712e-49b3-8ad8-db96f89baa82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.859838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6267f-712e-49b3-8ad8-db96f89baa82 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.860121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6267f-712e-49b3-8ad8-db96f89baa82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.860397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e6267f-712e-49b3-8ad8-db96f89baa82 admin admin] PATH_INFO: `/v3/projects/9b34d8e1c64742f59167ceb66c5adb64/users/3f97ae56f8374832a7e9805f930af649/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.874870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6267f-712e-49b3-8ad8-db96f89baa82 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b34d8e1c64742f59167ceb66c5adb64, user_id=3f97ae56f8374832a7e9805f930af649, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.877531 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e6267f-712e-49b3-8ad8-db96f89baa82 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.891509 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1313/2623] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:49 2022] PUT /identity/v3/projects/9b34d8e1c64742f59167ceb66c5adb64/users/3f97ae56f8374832a7e9805f930af649/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:08:49.895478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a78937-2e99-4361-9996-73723627ea8d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.906499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a78937-2e99-4361-9996-73723627ea8d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.962184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8dc0ef-226b-4f35-93ad-d44507296a32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.962809 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dc0ef-226b-4f35-93ad-d44507296a32 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.963092 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dc0ef-226b-4f35-93ad-d44507296a32 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.963367 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8dc0ef-226b-4f35-93ad-d44507296a32 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.965009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8dc0ef-226b-4f35-93ad-d44507296a32 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:49.966632 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8dc0ef-226b-4f35-93ad-d44507296a32 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:49.976553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1311/2624] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:49.980912 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6267f-712e-49b3-8ad8-db96f89baa82 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:49.991164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e6267f-712e-49b3-8ad8-db96f89baa82 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:49.995894 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac455f-7944-494e-bf37-8b12a94ab948 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:49.996767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac455f-7944-494e-bf37-8b12a94ab948 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:49.997051 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac455f-7944-494e-bf37-8b12a94ab948 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:49.997328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bac455f-7944-494e-bf37-8b12a94ab948 admin admin] PATH_INFO: `/v3/projects/9b34d8e1c64742f59167ceb66c5adb64/users/3f97ae56f8374832a7e9805f930af649/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:49.999330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac455f-7944-494e-bf37-8b12a94ab948 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b34d8e1c64742f59167ceb66c5adb64, user_id=3f97ae56f8374832a7e9805f930af649, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:50.002002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bac455f-7944-494e-bf37-8b12a94ab948 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:50.012696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1314/2625] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:49 2022] PUT /identity/v3/projects/9b34d8e1c64742f59167ceb66c5adb64/users/3f97ae56f8374832a7e9805f930af649/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:50.022761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72a574-1188-4975-9fec-981ceb926dbd None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:50.023216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72a574-1188-4975-9fec-981ceb926dbd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:50.023302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72a574-1188-4975-9fec-981ceb926dbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:50.050297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-7c72a574-1188-4975-9fec-981ceb926dbd None None] MFA Rules not processed for user `3f97ae56f8374832a7e9805f930af649`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:50.051493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7c72a574-1188-4975-9fec-981ceb926dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:50.117022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1312/2626] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:50 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:50.129349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac455f-7944-494e-bf37-8b12a94ab948 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:50.131266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0bac455f-7944-494e-bf37-8b12a94ab948 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:50.189054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bac455f-7944-494e-bf37-8b12a94ab948 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:50.192831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bf27b-7f1c-412a-987c-809a9556b843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:50.193477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bf27b-7f1c-412a-987c-809a9556b843 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:50.193800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bf27b-7f1c-412a-987c-809a9556b843 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:50.194083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8bf27b-7f1c-412a-987c-809a9556b843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:50.249115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bf27b-7f1c-412a-987c-809a9556b843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:50.250512 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8bf27b-7f1c-412a-987c-809a9556b843 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:50.259237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1315/2627] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:50 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:51.745568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72a574-1188-4975-9fec-981ceb926dbd None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:51.746973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-7c72a574-1188-4975-9fec-981ceb926dbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:51.759022 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72a574-1188-4975-9fec-981ceb926dbd None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:51.763085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2260a-b9a7-4947-8a48-ddda2bee0c04 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:51.763726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2260a-b9a7-4947-8a48-ddda2bee0c04 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:51.764019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2260a-b9a7-4947-8a48-ddda2bee0c04 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:51.764296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf2260a-b9a7-4947-8a48-ddda2bee0c04 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:51.816711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:51.821295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2260a-b9a7-4947-8a48-ddda2bee0c04 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:51.823790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf2260a-b9a7-4947-8a48-ddda2bee0c04 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:51.833008 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1313/2628] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:52.344033 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bf27b-7f1c-412a-987c-809a9556b843 service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:52.352914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8bf27b-7f1c-412a-987c-809a9556b843 service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:52.357044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c7a81d-2e0c-44e6-b14d-6784da4bc6b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:52.357702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7a81d-2e0c-44e6-b14d-6784da4bc6b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:52.357987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7a81d-2e0c-44e6-b14d-6784da4bc6b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:52.358280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c7a81d-2e0c-44e6-b14d-6784da4bc6b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:52.367146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7a81d-2e0c-44e6-b14d-6784da4bc6b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:52.368552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c7a81d-2e0c-44e6-b14d-6784da4bc6b5 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:52.378608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1316/2629] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:52 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:56.199583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2260a-b9a7-4947-8a48-ddda2bee0c04 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:56.261302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf2260a-b9a7-4947-8a48-ddda2bee0c04 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:56.266280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:56.267156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:56.267443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:56.267724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] PATH_INFO: `/v3/users/1546f2768de34bf9855f5f6d80fb2671` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:56.268915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1546f2768de34bf9855f5f6d80fb2671)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:56.270883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:56.317330 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1546f2768de34bf9855f5f6d80fb2671'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:56.329609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1546f2768de34bf9855f5f6d80fb2671'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:56.336436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1546f2768de34bf9855f5f6d80fb2671'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:56.337118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1546f2768de34bf9855f5f6d80fb2671'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:56.337747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1546f2768de34bf9855f5f6d80fb2671'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:56.351598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1314/2630] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:56 2022] DELETE /identity/v3/users/1546f2768de34bf9855f5f6d80fb2671 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:56.356136 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4ff45-5d53-4dfb-9627-7f9d9608d310 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:56.356566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4ff45-5d53-4dfb-9627-7f9d9608d310 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:56.356707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19a4ff45-5d53-4dfb-9627-7f9d9608d310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:56.374663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-19a4ff45-5d53-4dfb-9627-7f9d9608d310 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:56.375860 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-19a4ff45-5d53-4dfb-9627-7f9d9608d310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:56.393719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:56.436097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1317/2631] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:56.445682 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:56.447150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:56.504004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c183ac-51a4-413b-bed4-fe60d195e86b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:56.507693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0136a98-ed61-49bf-8256-781027c3b522 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:56.508337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0136a98-ed61-49bf-8256-781027c3b522 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:56.508619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0136a98-ed61-49bf-8256-781027c3b522 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:56.508923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0136a98-ed61-49bf-8256-781027c3b522 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:56.518103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0136a98-ed61-49bf-8256-781027c3b522 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:56.519535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0136a98-ed61-49bf-8256-781027c3b522 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:56.529419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1315/2632] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:08:56.789391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4ff45-5d53-4dfb-9627-7f9d9608d310 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:56.790941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-19a4ff45-5d53-4dfb-9627-7f9d9608d310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:56.803288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19a4ff45-5d53-4dfb-9627-7f9d9608d310 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:56.807833 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:56.808687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:56.808969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:56.809243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] PATH_INFO: `/v3/projects/61a09f54e29842bcb919c1cb18fd5e6b` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:56.810475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] RBAC: Authorizing `identity:delete_project(project_id=61a09f54e29842bcb919c1cb18fd5e6b)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:56.812371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:56.858972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '61a09f54e29842bcb919c1cb18fd5e6b', 'request_id': 'req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:56.865046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '61a09f54e29842bcb919c1cb18fd5e6b', 'request_id': 'req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:56.870503 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '61a09f54e29842bcb919c1cb18fd5e6b', 'request_id': 'req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:56.882441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] The token cache is being invalidate because project 61a09f54e29842bcb919c1cb18fd5e6b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:56.882668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:56.883283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:56.884804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1318/2633] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:56 2022] DELETE /identity/v3/projects/61a09f54e29842bcb919c1cb18fd5e6b => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:56.889294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0136a98-ed61-49bf-8256-781027c3b522 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:56.899283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0136a98-ed61-49bf-8256-781027c3b522 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:56.911285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:56.960381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:56.961263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:56.961575 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:56.961857 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] PATH_INFO: `/v3/users/aa073a1a2e4944f1b74053c8dd109fe0` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:56.963030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] RBAC: Authorizing `identity:delete_user(user_id=aa073a1a2e4944f1b74053c8dd109fe0)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:56.964983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:57.010880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'aa073a1a2e4944f1b74053c8dd109fe0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.023156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'aa073a1a2e4944f1b74053c8dd109fe0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.029784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'aa073a1a2e4944f1b74053c8dd109fe0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.030465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'aa073a1a2e4944f1b74053c8dd109fe0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.031129 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'aa073a1a2e4944f1b74053c8dd109fe0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.043553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1316/2634] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:56 2022] DELETE /identity/v3/users/aa073a1a2e4944f1b74053c8dd109fe0 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:57.321299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:57.321299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:57.332869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fffad9-d5cd-4e44-a6aa-9e5afaac4d92 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:57.389453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:57.390338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:57.390624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:57.390921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] PATH_INFO: `/v3/projects/95288af0d57e410fb434282f4cfdefa1` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:57.392156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=95288af0d57e410fb434282f4cfdefa1)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:57.394081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:57.440567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '95288af0d57e410fb434282f4cfdefa1', 'request_id': 'req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.441660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:57.448194 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '95288af0d57e410fb434282f4cfdefa1', 'request_id': 'req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.454481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '95288af0d57e410fb434282f4cfdefa1', 'request_id': 'req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.466547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] The token cache is being invalidate because project 95288af0d57e410fb434282f4cfdefa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:57.466834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.467489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.468995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1319/2635] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:57 2022] DELETE /identity/v3/projects/95288af0d57e410fb434282f4cfdefa1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:57.473094 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:57.474772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:57.487872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981c794f-8767-451d-bab5-b096fc32fbad admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:57.545058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:57.545958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:57.546244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:57.546523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] PATH_INFO: `/v3/users/3f97ae56f8374832a7e9805f930af649` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:57.547741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3f97ae56f8374832a7e9805f930af649)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:57.549711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:57.587175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:57.597271 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3f97ae56f8374832a7e9805f930af649'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.610647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3f97ae56f8374832a7e9805f930af649'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.617420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3f97ae56f8374832a7e9805f930af649'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.618124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3f97ae56f8374832a7e9805f930af649'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.618741 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6d870689-b290-47f4-b7ef-f4e5149b582b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3f97ae56f8374832a7e9805f930af649'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:57.632657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1317/2636] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:08:57 2022] DELETE /identity/v3/users/3f97ae56f8374832a7e9805f930af649 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:57.898074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:57.899577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:57.913783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e34d46-d894-4f6f-8664-a7f110d9d0f5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:57.971230 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:57.972198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:57.972491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:57.972768 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] PATH_INFO: `/v3/projects/9b34d8e1c64742f59167ceb66c5adb64` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:57.973973 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b34d8e1c64742f59167ceb66c5adb64)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:57.976143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:58.021943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b34d8e1c64742f59167ceb66c5adb64', 'request_id': 'req-39489530-d3e8-4fd1-814f-e05ac2f79f42'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:58.027966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b34d8e1c64742f59167ceb66c5adb64', 'request_id': 'req-39489530-d3e8-4fd1-814f-e05ac2f79f42'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:58.034137 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b34d8e1c64742f59167ceb66c5adb64', 'request_id': 'req-39489530-d3e8-4fd1-814f-e05ac2f79f42'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:58.046718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] The token cache is being invalidate because project 9b34d8e1c64742f59167ceb66c5adb64 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:08:58.047007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:58.047634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-39489530-d3e8-4fd1-814f-e05ac2f79f42 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:08:58.049158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1320/2637] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:08:57 2022] DELETE /identity/v3/projects/9b34d8e1c64742f59167ceb66c5adb64 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:58.054897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564f1ab-c5df-409b-927f-07092df376cd None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:58.055188 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564f1ab-c5df-409b-927f-07092df376cd None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:58.055468 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6564f1ab-c5df-409b-927f-07092df376cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:58.072773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6564f1ab-c5df-409b-927f-07092df376cd None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:58.074055 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6564f1ab-c5df-409b-927f-07092df376cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:58.132912 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1318/2638] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:58.142296 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f8c65-f33f-470e-8e4a-1f9ba26cdc47 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:58.142584 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f8c65-f33f-470e-8e4a-1f9ba26cdc47 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:58.142886 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f8c65-f33f-470e-8e4a-1f9ba26cdc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:58.161559 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-025f8c65-f33f-470e-8e4a-1f9ba26cdc47 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:58.162700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-025f8c65-f33f-470e-8e4a-1f9ba26cdc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:58.171260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1321/2639] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:58.175543 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564f1ab-c5df-409b-927f-07092df376cd None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:58.185820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6564f1ab-c5df-409b-927f-07092df376cd None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:58.190629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedafe70-5c3a-46a0-b9bf-301b756cf927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:58.191287 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedafe70-5c3a-46a0-b9bf-301b756cf927 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:58.191572 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedafe70-5c3a-46a0-b9bf-301b756cf927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:58.191851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bedafe70-5c3a-46a0-b9bf-301b756cf927 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:58.192583 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedafe70-5c3a-46a0-b9bf-301b756cf927 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:08:58.192891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedafe70-5c3a-46a0-b9bf-301b756cf927 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:58.194903 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bedafe70-5c3a-46a0-b9bf-301b756cf927 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:58.207144 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1319/2640] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:08:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:08:58.211676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f8c65-f33f-470e-8e4a-1f9ba26cdc47 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:58.221810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f8c65-f33f-470e-8e4a-1f9ba26cdc47 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:58.226739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed89e7-8fd5-4059-b81b-664b63635c8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:58.227383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed89e7-8fd5-4059-b81b-664b63635c8d admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:58.227666 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed89e7-8fd5-4059-b81b-664b63635c8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:58.227941 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baed89e7-8fd5-4059-b81b-664b63635c8d admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:58.228631 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed89e7-8fd5-4059-b81b-664b63635c8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:58.230636 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baed89e7-8fd5-4059-b81b-664b63635c8d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:58.255049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1322/2641] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:08:58 2022] POST /identity/v3/projects => generated 375 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:58.259402 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedafe70-5c3a-46a0-b9bf-301b756cf927 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:58.270178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bedafe70-5c3a-46a0-b9bf-301b756cf927 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:58.327927 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89ad51-8f9f-4ed4-8323-c5dbbed6055f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:58.328600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89ad51-8f9f-4ed4-8323-c5dbbed6055f admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:58.328886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89ad51-8f9f-4ed4-8323-c5dbbed6055f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:58.329160 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b89ad51-8f9f-4ed4-8323-c5dbbed6055f admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:58.329881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89ad51-8f9f-4ed4-8323-c5dbbed6055f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:58.331827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b89ad51-8f9f-4ed4-8323-c5dbbed6055f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:58.355351 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1320/2642] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:08:58 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:08:58.356472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed89e7-8fd5-4059-b81b-664b63635c8d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:58.366888 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baed89e7-8fd5-4059-b81b-664b63635c8d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:58.371482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34976529-c5b4-4462-a76a-b067534fd2fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:58.372136 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34976529-c5b4-4462-a76a-b067534fd2fe admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:58.372422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34976529-c5b4-4462-a76a-b067534fd2fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:58.372707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34976529-c5b4-4462-a76a-b067534fd2fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:58.373533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34976529-c5b4-4462-a76a-b067534fd2fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:58.375146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34976529-c5b4-4462-a76a-b067534fd2fe admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:58.385420 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1323/2643] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:08:58.389893 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89ad51-8f9f-4ed4-8323-c5dbbed6055f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:58.400158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b89ad51-8f9f-4ed4-8323-c5dbbed6055f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:58.405723 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5066d368-255d-4ce1-bef3-f38445a06cdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:58.406438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066d368-255d-4ce1-bef3-f38445a06cdc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:58.406724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066d368-255d-4ce1-bef3-f38445a06cdc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:58.407003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5066d368-255d-4ce1-bef3-f38445a06cdc admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:58.407736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066d368-255d-4ce1-bef3-f38445a06cdc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:58.409338 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5066d368-255d-4ce1-bef3-f38445a06cdc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:58.419352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1321/2644] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:08:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:08:58.423916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34976529-c5b4-4462-a76a-b067534fd2fe admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:58.434260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34976529-c5b4-4462-a76a-b067534fd2fe admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:58.439576 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb38bd4-3c33-46f7-bb51-3d5458303ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:58.440453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb38bd4-3c33-46f7-bb51-3d5458303ff1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:58.440737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb38bd4-3c33-46f7-bb51-3d5458303ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:58.441012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb38bd4-3c33-46f7-bb51-3d5458303ff1 admin admin] PATH_INFO: `/v3/projects/f1fe5b769bb1438398b8605fa7e71d93/users/a49d0713abd747eb9f30f64ab1d73cb5/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:58.455529 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb38bd4-3c33-46f7-bb51-3d5458303ff1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f1fe5b769bb1438398b8605fa7e71d93, user_id=a49d0713abd747eb9f30f64ab1d73cb5, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:58.458208 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb38bd4-3c33-46f7-bb51-3d5458303ff1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:58.471342 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1324/2645] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:08:58 2022] PUT /identity/v3/projects/f1fe5b769bb1438398b8605fa7e71d93/users/a49d0713abd747eb9f30f64ab1d73cb5/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:08:58.481823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d065a1-cbaa-4a0e-bcc6-dae328fa554f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:58.482879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d065a1-cbaa-4a0e-bcc6-dae328fa554f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:58.483045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d065a1-cbaa-4a0e-bcc6-dae328fa554f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:58.509072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-89d065a1-cbaa-4a0e-bcc6-dae328fa554f None None] MFA Rules not processed for user `a49d0713abd747eb9f30f64ab1d73cb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:08:58.510312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-89d065a1-cbaa-4a0e-bcc6-dae328fa554f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:58.518705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:58.578116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1322/2646] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:08:58 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:08:58.588787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb38bd4-3c33-46f7-bb51-3d5458303ff1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:08:58.590476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3bb38bd4-3c33-46f7-bb51-3d5458303ff1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:08:58.597783 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:08:58.649013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb38bd4-3c33-46f7-bb51-3d5458303ff1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:08:58.652648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c04c1ac-0faf-47fa-b77d-85edbda8aa9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:08:58.653315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04c1ac-0faf-47fa-b77d-85edbda8aa9b service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:08:58.653622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04c1ac-0faf-47fa-b77d-85edbda8aa9b service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:08:58.654462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c04c1ac-0faf-47fa-b77d-85edbda8aa9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:08:58.662962 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04c1ac-0faf-47fa-b77d-85edbda8aa9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:08:58.664443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c04c1ac-0faf-47fa-b77d-85edbda8aa9b service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:08:58.674493 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1325/2647] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:08:58 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:09:08.575209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=23 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:09:08.647736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d065a1-cbaa-4a0e-bcc6-dae328fa554f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:09:08.669297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=24 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:09:08.734677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d065a1-cbaa-4a0e-bcc6-dae328fa554f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:09:08.739199 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:09:08.740065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:09:08.740347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:09:08.740621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] PATH_INFO: `/v3/users/a49d0713abd747eb9f30f64ab1d73cb5` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:09:08.741824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a49d0713abd747eb9f30f64ab1d73cb5)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:09:08.743768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:09:08.793736 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a49d0713abd747eb9f30f64ab1d73cb5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:09:08.806062 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a49d0713abd747eb9f30f64ab1d73cb5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:09:08.813158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a49d0713abd747eb9f30f64ab1d73cb5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:09:08.813873 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a49d0713abd747eb9f30f64ab1d73cb5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:09:08.814478 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a49d0713abd747eb9f30f64ab1d73cb5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:09:08.827275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1323/2648] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:09:08 2022] DELETE /identity/v3/users/a49d0713abd747eb9f30f64ab1d73cb5 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:09:08.831933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6582dd-8cfa-46d2-949e-2e33c9de739f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:09:08.832267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6582dd-8cfa-46d2-949e-2e33c9de739f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:09:08.832547 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd6582dd-8cfa-46d2-949e-2e33c9de739f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:09:08.864734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-cd6582dd-8cfa-46d2-949e-2e33c9de739f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:09:08.865982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cd6582dd-8cfa-46d2-949e-2e33c9de739f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:09:08.925828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1326/2649] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:09:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:09:08.935085 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:09:08.936637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:09:08.993603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4c6643b-0ba7-485a-9786-2106a7237218 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:09:08.997189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17202473-ef95-4e64-a7c5-74b8b8ec0e3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:09:08.997857 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17202473-ef95-4e64-a7c5-74b8b8ec0e3c service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:09:08.998697 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17202473-ef95-4e64-a7c5-74b8b8ec0e3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:09:08.998976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17202473-ef95-4e64-a7c5-74b8b8ec0e3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:09:09.007876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17202473-ef95-4e64-a7c5-74b8b8ec0e3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:09:09.009301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17202473-ef95-4e64-a7c5-74b8b8ec0e3c service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:09:09.019555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1324/2650] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:09:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:09:09.285405 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6582dd-8cfa-46d2-949e-2e33c9de739f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:09:09.286872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-cd6582dd-8cfa-46d2-949e-2e33c9de739f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:09:09.301018 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd6582dd-8cfa-46d2-949e-2e33c9de739f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:09:09.306608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:09:09.307600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:09:09.308589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:09:09.308868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] PATH_INFO: `/v3/projects/f1fe5b769bb1438398b8605fa7e71d93` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:09:09.310265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f1fe5b769bb1438398b8605fa7e71d93)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:09:09.312990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:09:09.361752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f1fe5b769bb1438398b8605fa7e71d93', 'request_id': 'req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:09:09.367662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f1fe5b769bb1438398b8605fa7e71d93', 'request_id': 'req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:09:09.373185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f1fe5b769bb1438398b8605fa7e71d93', 'request_id': 'req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:09:09.385997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] The token cache is being invalidate because project f1fe5b769bb1438398b8605fa7e71d93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:09:09.386288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:09:09.386907 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:09:09.388470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1327/2651] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:09:09 2022] DELETE /identity/v3/projects/f1fe5b769bb1438398b8605fa7e71d93 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:09:19.015241 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:09:19.382799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=59,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:09:55.819869 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17202473-ef95-4e64-a7c5-74b8b8ec0e3c service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:09:55.821636 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-17202473-ef95-4e64-a7c5-74b8b8ec0e3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:09:55.906343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17202473-ef95-4e64-a7c5-74b8b8ec0e3c service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:09:55.994565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e3a52-2174-4153-b3ce-10b3da346c05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:09:55.995252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e3a52-2174-4153-b3ce-10b3da346c05 service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:09:55.995535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e3a52-2174-4153-b3ce-10b3da346c05 service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:09:55.995812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3e3a52-2174-4153-b3ce-10b3da346c05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:09:55.997799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ae3e3a52-2174-4153-b3ce-10b3da346c05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:09:56.011138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e3a52-2174-4153-b3ce-10b3da346c05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:09:56.012677 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae3e3a52-2174-4153-b3ce-10b3da346c05 service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:09:56.025584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1325/2652] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:09:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:00.743829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:00.745245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:00.801911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d255ae6-ba34-4e3a-9cdf-eaef6ceb2d97 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:00.806223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacd0e3-ce23-4108-8f66-baec949f33ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:00.806862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacd0e3-ce23-4108-8f66-baec949f33ea service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:00.807145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacd0e3-ce23-4108-8f66-baec949f33ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:00.807424 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eacd0e3-ce23-4108-8f66-baec949f33ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:00.809159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3eacd0e3-ce23-4108-8f66-baec949f33ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:00.879187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacd0e3-ce23-4108-8f66-baec949f33ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:00.880622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eacd0e3-ce23-4108-8f66-baec949f33ea service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:00.890415 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1328/2653] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:10:04.056672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e3a52-2174-4153-b3ce-10b3da346c05 service nova] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:04.058627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ae3e3a52-2174-4153-b3ce-10b3da346c05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:04.072949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3e3a52-2174-4153-b3ce-10b3da346c05 service nova] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:04.077593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:04.078462 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.078743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.079016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] PATH_INFO: `/v3/users/0d1cf9b55ce04cf997bf23ad73222814` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.080196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0d1cf9b55ce04cf997bf23ad73222814)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:04.082162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:04.129310 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0d1cf9b55ce04cf997bf23ad73222814'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:04.141087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0d1cf9b55ce04cf997bf23ad73222814'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:04.148593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0d1cf9b55ce04cf997bf23ad73222814'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:04.149262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0d1cf9b55ce04cf997bf23ad73222814'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:04.149881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-3c06116c-f255-4019-99eb-001eaaa2a8e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0d1cf9b55ce04cf997bf23ad73222814'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:04.154463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:04.166895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1326/2654] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:10:04 2022] DELETE /identity/v3/users/0d1cf9b55ce04cf997bf23ad73222814 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:10:04.340371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacd0e3-ce23-4108-8f66-baec949f33ea service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:04.350979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eacd0e3-ce23-4108-8f66-baec949f33ea service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:04.413645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:04.414512 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.414797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.415080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] PATH_INFO: `/v3/projects/b2b576df300148ad8d8d7dd3f8335f8e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.416741 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2b576df300148ad8d8d7dd3f8335f8e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:04.419699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:04.468790 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2b576df300148ad8d8d7dd3f8335f8e', 'request_id': 'req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:04.470597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:04.476301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2b576df300148ad8d8d7dd3f8335f8e', 'request_id': 'req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:04.482645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2b576df300148ad8d8d7dd3f8335f8e', 'request_id': 'req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:04.494999 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] The token cache is being invalidate because project b2b576df300148ad8d8d7dd3f8335f8e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:10:04.495285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:04.495916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0347e1de-453e-4cc7-aedb-8af1ef8b3de5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:04.497452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1329/2655] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:10:04 2022] DELETE /identity/v3/projects/b2b576df300148ad8d8d7dd3f8335f8e => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:04.502500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414b65-40a4-4a55-92e3-bcac65286041 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.502799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414b65-40a4-4a55-92e3-bcac65286041 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.503106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4414b65-40a4-4a55-92e3-bcac65286041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.522475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b4414b65-40a4-4a55-92e3-bcac65286041 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:04.523689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b4414b65-40a4-4a55-92e3-bcac65286041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:04.582347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1327/2656] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:04.590277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ffe18-36fb-444a-8d47-99269ff2dcc2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.590568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ffe18-36fb-444a-8d47-99269ff2dcc2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.590849 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-224ffe18-36fb-444a-8d47-99269ff2dcc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.609838 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-224ffe18-36fb-444a-8d47-99269ff2dcc2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:04.611055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-224ffe18-36fb-444a-8d47-99269ff2dcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:04.620138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1330/2657] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:04.624377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4414b65-40a4-4a55-92e3-bcac65286041 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:04.634306 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4414b65-40a4-4a55-92e3-bcac65286041 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:04.638915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efba09b-fc7c-4589-8d1f-fc20e1559048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:04.639570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efba09b-fc7c-4589-8d1f-fc20e1559048 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.639852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efba09b-fc7c-4589-8d1f-fc20e1559048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.640131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0efba09b-fc7c-4589-8d1f-fc20e1559048 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.640850 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efba09b-fc7c-4589-8d1f-fc20e1559048 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:10:04.641156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efba09b-fc7c-4589-8d1f-fc20e1559048 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:04.642788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0efba09b-fc7c-4589-8d1f-fc20e1559048 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:04.654435 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1328/2658] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:10:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:10:04.658864 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ffe18-36fb-444a-8d47-99269ff2dcc2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:04.668967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-224ffe18-36fb-444a-8d47-99269ff2dcc2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:04.673590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e08e8-7d37-4045-9f98-85e5bd5e8ee3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:04.674227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e08e8-7d37-4045-9f98-85e5bd5e8ee3 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.674505 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e08e8-7d37-4045-9f98-85e5bd5e8ee3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.674776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d11e08e8-7d37-4045-9f98-85e5bd5e8ee3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.675458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e08e8-7d37-4045-9f98-85e5bd5e8ee3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:04.677402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d11e08e8-7d37-4045-9f98-85e5bd5e8ee3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:04.703765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1331/2659] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:04 2022] POST /identity/v3/projects => generated 373 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:04.707813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efba09b-fc7c-4589-8d1f-fc20e1559048 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:04.719063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0efba09b-fc7c-4589-8d1f-fc20e1559048 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:04.774899 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c576feb-2145-43b9-9bd4-80d349d10b35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:04.775564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c576feb-2145-43b9-9bd4-80d349d10b35 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.775844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c576feb-2145-43b9-9bd4-80d349d10b35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.776116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c576feb-2145-43b9-9bd4-80d349d10b35 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.776800 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c576feb-2145-43b9-9bd4-80d349d10b35 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:04.778747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c576feb-2145-43b9-9bd4-80d349d10b35 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:04.798602 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1329/2660] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:04 2022] POST /identity/v3/users => generated 287 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:10:04.802719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e08e8-7d37-4045-9f98-85e5bd5e8ee3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:04.813205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d11e08e8-7d37-4045-9f98-85e5bd5e8ee3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:04.817813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e4fb3-a65d-4992-a742-b815807aa103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:04.818461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e4fb3-a65d-4992-a742-b815807aa103 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.818879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e4fb3-a65d-4992-a742-b815807aa103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.819275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83e4fb3-a65d-4992-a742-b815807aa103 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.820013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e4fb3-a65d-4992-a742-b815807aa103 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:04.821600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83e4fb3-a65d-4992-a742-b815807aa103 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:04.831909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1332/2661] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:04.836315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c576feb-2145-43b9-9bd4-80d349d10b35 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:04.846034 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c576feb-2145-43b9-9bd4-80d349d10b35 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:04.851219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4942e5ff-3dde-435a-aeef-28318e91bcf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:04.851846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942e5ff-3dde-435a-aeef-28318e91bcf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.852124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942e5ff-3dde-435a-aeef-28318e91bcf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.852397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4942e5ff-3dde-435a-aeef-28318e91bcf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.853113 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942e5ff-3dde-435a-aeef-28318e91bcf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:04.855089 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4942e5ff-3dde-435a-aeef-28318e91bcf5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:04.864908 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1330/2662] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:04 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:04.869111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e4fb3-a65d-4992-a742-b815807aa103 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:04.879199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83e4fb3-a65d-4992-a742-b815807aa103 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:04.884226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ba043-514f-4066-9279-cf307263d355 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:04.885095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ba043-514f-4066-9279-cf307263d355 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.885382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ba043-514f-4066-9279-cf307263d355 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.885691 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147ba043-514f-4066-9279-cf307263d355 admin admin] PATH_INFO: `/v3/projects/dfdc42043200455faee589dcf1410a36/users/0a2973bc7bf0431980b9f3cb5a759479/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.900574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ba043-514f-4066-9279-cf307263d355 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfdc42043200455faee589dcf1410a36, user_id=0a2973bc7bf0431980b9f3cb5a759479, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:04.903271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-147ba043-514f-4066-9279-cf307263d355 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:04.924561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1333/2663] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:04 2022] PUT /identity/v3/projects/dfdc42043200455faee589dcf1410a36/users/0a2973bc7bf0431980b9f3cb5a759479/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:04.934860 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c61a86-f8e6-4588-9ef1-7d9f72837660 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:04.935150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c61a86-f8e6-4588-9ef1-7d9f72837660 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:04.935424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c61a86-f8e6-4588-9ef1-7d9f72837660 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:04.963529 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f5c61a86-f8e6-4588-9ef1-7d9f72837660 None None] MFA Rules not processed for user `0a2973bc7bf0431980b9f3cb5a759479`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:04.964759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f5c61a86-f8e6-4588-9ef1-7d9f72837660 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:05.011798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:05.030243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1331/2664] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:04 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:10:05.034703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ba043-514f-4066-9279-cf307263d355 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:05.095950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-147ba043-514f-4066-9279-cf307263d355 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:05.100538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fefa2-793f-44aa-a731-91c3bf4b9f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:05.101182 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fefa2-793f-44aa-a731-91c3bf4b9f41 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:05.102033 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fefa2-793f-44aa-a731-91c3bf4b9f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:05.102315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-174fefa2-793f-44aa-a731-91c3bf4b9f41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:05.103003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fefa2-793f-44aa-a731-91c3bf4b9f41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:05.105151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-174fefa2-793f-44aa-a731-91c3bf4b9f41 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:05.129023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1334/2665] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:05 2022] POST /identity/v3/projects => generated 371 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:05.133900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c61a86-f8e6-4588-9ef1-7d9f72837660 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:05.144595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c61a86-f8e6-4588-9ef1-7d9f72837660 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:05.199707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715ad4d-e8d2-458d-a86e-fd941d8f065a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:05.200365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715ad4d-e8d2-458d-a86e-fd941d8f065a admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:05.200646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715ad4d-e8d2-458d-a86e-fd941d8f065a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:05.200922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a715ad4d-e8d2-458d-a86e-fd941d8f065a admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:05.201619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715ad4d-e8d2-458d-a86e-fd941d8f065a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:05.203558 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a715ad4d-e8d2-458d-a86e-fd941d8f065a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:05.222917 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1332/2666] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:05 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:05.227013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fefa2-793f-44aa-a731-91c3bf4b9f41 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:05.237488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-174fefa2-793f-44aa-a731-91c3bf4b9f41 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:05.242160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a6cdf-6e00-438f-bf00-486e45c4a26b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:05.242809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a6cdf-6e00-438f-bf00-486e45c4a26b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:05.243098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a6cdf-6e00-438f-bf00-486e45c4a26b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:05.243379 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a6cdf-6e00-438f-bf00-486e45c4a26b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:05.244108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a6cdf-6e00-438f-bf00-486e45c4a26b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:05.245726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f6a6cdf-6e00-438f-bf00-486e45c4a26b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:05.256588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1335/2667] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:05.260767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715ad4d-e8d2-458d-a86e-fd941d8f065a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:05.270916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a715ad4d-e8d2-458d-a86e-fd941d8f065a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:05.276345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595d66f-c5e0-4759-b69a-423d0303e1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:05.277248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595d66f-c5e0-4759-b69a-423d0303e1dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:05.277592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595d66f-c5e0-4759-b69a-423d0303e1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:05.277879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e595d66f-c5e0-4759-b69a-423d0303e1dc admin admin] PATH_INFO: `/v3/projects/46450871949944509c4acc822c20a2cf/users/fea86cf17f2642fe8ca45cee98d9466e/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:05.292408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595d66f-c5e0-4759-b69a-423d0303e1dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=46450871949944509c4acc822c20a2cf, user_id=fea86cf17f2642fe8ca45cee98d9466e, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:05.295109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e595d66f-c5e0-4759-b69a-423d0303e1dc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:05.308358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1333/2668] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:05 2022] PUT /identity/v3/projects/46450871949944509c4acc822c20a2cf/users/fea86cf17f2642fe8ca45cee98d9466e/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:05.312233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a6cdf-6e00-438f-bf00-486e45c4a26b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:05.323720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a6cdf-6e00-438f-bf00-486e45c4a26b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:05.369877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:05.381727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8606edaa-65e1-4ac5-b97b-2c6946f24aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:05.382402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606edaa-65e1-4ac5-b97b-2c6946f24aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:05.382688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606edaa-65e1-4ac5-b97b-2c6946f24aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:05.382972 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8606edaa-65e1-4ac5-b97b-2c6946f24aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:05.383694 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606edaa-65e1-4ac5-b97b-2c6946f24aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:05.385282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8606edaa-65e1-4ac5-b97b-2c6946f24aa2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:05.395217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1336/2669] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:05 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:05.399927 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595d66f-c5e0-4759-b69a-423d0303e1dc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:05.410445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e595d66f-c5e0-4759-b69a-423d0303e1dc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:05.415274 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29ca70-c6f8-41fc-a4fc-3662c94ac595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:05.416150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29ca70-c6f8-41fc-a4fc-3662c94ac595 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:05.416437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29ca70-c6f8-41fc-a4fc-3662c94ac595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:05.416716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce29ca70-c6f8-41fc-a4fc-3662c94ac595 admin admin] PATH_INFO: `/v3/projects/46450871949944509c4acc822c20a2cf/users/fea86cf17f2642fe8ca45cee98d9466e/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:05.418713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29ca70-c6f8-41fc-a4fc-3662c94ac595 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46450871949944509c4acc822c20a2cf, user_id=fea86cf17f2642fe8ca45cee98d9466e, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:05.422176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce29ca70-c6f8-41fc-a4fc-3662c94ac595 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:05.434381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1334/2670] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:05 2022] PUT /identity/v3/projects/46450871949944509c4acc822c20a2cf/users/fea86cf17f2642fe8ca45cee98d9466e/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:05.446412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5972a-0875-4720-a656-935f0bc963de None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:05.446710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5972a-0875-4720-a656-935f0bc963de None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:05.446991 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9f5972a-0875-4720-a656-935f0bc963de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:05.473286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a9f5972a-0875-4720-a656-935f0bc963de None None] MFA Rules not processed for user `fea86cf17f2642fe8ca45cee98d9466e`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:05.474523 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a9f5972a-0875-4720-a656-935f0bc963de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:05.542044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1337/2671] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:05 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:05.553584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29ca70-c6f8-41fc-a4fc-3662c94ac595 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:05.555546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ce29ca70-c6f8-41fc-a4fc-3662c94ac595 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:05.631914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce29ca70-c6f8-41fc-a4fc-3662c94ac595 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:05.635588 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eacd7-a9ea-4315-b17f-63da76c74686 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:05.636237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eacd7-a9ea-4315-b17f-63da76c74686 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:05.636521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eacd7-a9ea-4315-b17f-63da76c74686 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:05.636799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752eacd7-a9ea-4315-b17f-63da76c74686 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:05.645777 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752eacd7-a9ea-4315-b17f-63da76c74686 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:05.647277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752eacd7-a9ea-4315-b17f-63da76c74686 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:05.656598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1335/2672] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:05 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:05.726241 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5972a-0875-4720-a656-935f0bc963de None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:05.736447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9f5972a-0875-4720-a656-935f0bc963de None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:05.740163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889541f-551b-4b2b-80ad-b5ed3b5d8c3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:05.741749 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889541f-551b-4b2b-80ad-b5ed3b5d8c3d service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:05.741949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889541f-551b-4b2b-80ad-b5ed3b5d8c3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:05.742227 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8889541f-551b-4b2b-80ad-b5ed3b5d8c3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:05.798703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889541f-551b-4b2b-80ad-b5ed3b5d8c3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:05.800069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8889541f-551b-4b2b-80ad-b5ed3b5d8c3d service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:05.809472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1338/2673] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:10:15.652386 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=79,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:15.805397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=68 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:23.983863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eacd7-a9ea-4315-b17f-63da76c74686 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:23.985252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-752eacd7-a9ea-4315-b17f-63da76c74686 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:24.043701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752eacd7-a9ea-4315-b17f-63da76c74686 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:24.047245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c84c2-8140-4cd7-b749-cfa13d989f95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:24.047888 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c84c2-8140-4cd7-b749-cfa13d989f95 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:24.048169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c84c2-8140-4cd7-b749-cfa13d989f95 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:24.048444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651c84c2-8140-4cd7-b749-cfa13d989f95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:24.050249 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-651c84c2-8140-4cd7-b749-cfa13d989f95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:24.112146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c84c2-8140-4cd7-b749-cfa13d989f95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:24.113599 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651c84c2-8140-4cd7-b749-cfa13d989f95 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:24.123299 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1336/2674] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:27.176827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889541f-551b-4b2b-80ad-b5ed3b5d8c3d service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:27.179458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8889541f-551b-4b2b-80ad-b5ed3b5d8c3d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:27.256480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c84c2-8140-4cd7-b749-cfa13d989f95 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:27.258556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-651c84c2-8140-4cd7-b749-cfa13d989f95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:27.278837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651c84c2-8140-4cd7-b749-cfa13d989f95 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:27.286882 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:27.287833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:27.288124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:27.288424 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] PATH_INFO: `/v3/users/343e480dd5ee4389ba430129cd7daaa4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:27.289684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] RBAC: Authorizing `identity:delete_user(user_id=343e480dd5ee4389ba430129cd7daaa4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:27.291971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:27.301162 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8889541f-551b-4b2b-80ad-b5ed3b5d8c3d service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:27.308598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8864cd-a770-4379-92d2-beb6188b52ff service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:27.309348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8864cd-a770-4379-92d2-beb6188b52ff service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:27.309951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8864cd-a770-4379-92d2-beb6188b52ff service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:27.310237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e8864cd-a770-4379-92d2-beb6188b52ff service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:27.314870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5e8864cd-a770-4379-92d2-beb6188b52ff service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:27.334219 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8864cd-a770-4379-92d2-beb6188b52ff service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:27.336345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e8864cd-a770-4379-92d2-beb6188b52ff service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:27.347364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1339/2675] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:27 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:27.353308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '343e480dd5ee4389ba430129cd7daaa4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:27.365722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '343e480dd5ee4389ba430129cd7daaa4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:27.373173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '343e480dd5ee4389ba430129cd7daaa4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:27.373890 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '343e480dd5ee4389ba430129cd7daaa4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:27.374566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-72f7386e-7ecd-4e33-8b27-8331ab08430c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '343e480dd5ee4389ba430129cd7daaa4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:27.388894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1337/2676] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:10:27 2022] DELETE /identity/v3/users/343e480dd5ee4389ba430129cd7daaa4 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:27.559098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8864cd-a770-4379-92d2-beb6188b52ff service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:27.576961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e8864cd-a770-4379-92d2-beb6188b52ff service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:27.637423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:27.638341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:27.638624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:27.638907 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] PATH_INFO: `/v3/projects/e6e0bccb45a24af3918f7db2c60de33d` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:27.640093 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e6e0bccb45a24af3918f7db2c60de33d)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:27.642029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:27.692120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e6e0bccb45a24af3918f7db2c60de33d', 'request_id': 'req-60eda71b-234c-4577-b69a-ef4282554c26'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:27.698422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e6e0bccb45a24af3918f7db2c60de33d', 'request_id': 'req-60eda71b-234c-4577-b69a-ef4282554c26'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:27.704121 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e6e0bccb45a24af3918f7db2c60de33d', 'request_id': 'req-60eda71b-234c-4577-b69a-ef4282554c26'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:27.716912 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] The token cache is being invalidate because project e6e0bccb45a24af3918f7db2c60de33d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:10:27.717206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:27.717975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-60eda71b-234c-4577-b69a-ef4282554c26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:27.719922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1340/2677] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:10:27 2022] DELETE /identity/v3/projects/e6e0bccb45a24af3918f7db2c60de33d => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:27.724855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98b77-792b-44d7-bef9-5d0a5f2192f6 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:27.725219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98b77-792b-44d7-bef9-5d0a5f2192f6 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:27.725610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef98b77-792b-44d7-bef9-5d0a5f2192f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:27.744983 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fef98b77-792b-44d7-bef9-5d0a5f2192f6 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:27.746214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fef98b77-792b-44d7-bef9-5d0a5f2192f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:27.763693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:27.812400 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1338/2678] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:27.820462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa55c7-48ca-4285-9491-f7044cee4f41 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:27.820819 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa55c7-48ca-4285-9491-f7044cee4f41 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:27.821105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfa55c7-48ca-4285-9491-f7044cee4f41 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:27.839050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7cfa55c7-48ca-4285-9491-f7044cee4f41 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:27.840289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7cfa55c7-48ca-4285-9491-f7044cee4f41 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:27.848496 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1341/2679] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:27.853944 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98b77-792b-44d7-bef9-5d0a5f2192f6 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:27.866752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef98b77-792b-44d7-bef9-5d0a5f2192f6 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:27.872128 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa659ed-1a14-41ef-bf41-e51ccbee463d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:27.872786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa659ed-1a14-41ef-bf41-e51ccbee463d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:27.873063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa659ed-1a14-41ef-bf41-e51ccbee463d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:27.873336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa659ed-1a14-41ef-bf41-e51ccbee463d admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:27.874118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa659ed-1a14-41ef-bf41-e51ccbee463d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:10:27.874426 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa659ed-1a14-41ef-bf41-e51ccbee463d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:27.876013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fa659ed-1a14-41ef-bf41-e51ccbee463d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:27.889562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1339/2680] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:10:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:10:27.892897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa55c7-48ca-4285-9491-f7044cee4f41 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:27.902658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfa55c7-48ca-4285-9491-f7044cee4f41 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:27.907647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51a28f-9355-4578-9c2b-254f0c3cc088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:27.908365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51a28f-9355-4578-9c2b-254f0c3cc088 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:27.908744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51a28f-9355-4578-9c2b-254f0c3cc088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:27.909041 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e51a28f-9355-4578-9c2b-254f0c3cc088 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:27.909844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51a28f-9355-4578-9c2b-254f0c3cc088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:27.911751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e51a28f-9355-4578-9c2b-254f0c3cc088 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:27.922730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:27.940607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1342/2681] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:27 2022] POST /identity/v3/projects => generated 403 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:27.944721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa659ed-1a14-41ef-bf41-e51ccbee463d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:27.954964 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa659ed-1a14-41ef-bf41-e51ccbee463d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:28.011414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594a927-a735-4e47-8fb7-922971522e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:28.012074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594a927-a735-4e47-8fb7-922971522e56 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:28.012354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594a927-a735-4e47-8fb7-922971522e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:28.012627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9594a927-a735-4e47-8fb7-922971522e56 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:28.013295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594a927-a735-4e47-8fb7-922971522e56 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:28.015231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9594a927-a735-4e47-8fb7-922971522e56 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:28.034563 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1340/2682] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:27 2022] POST /identity/v3/users => generated 302 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:28.038719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51a28f-9355-4578-9c2b-254f0c3cc088 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:28.049780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e51a28f-9355-4578-9c2b-254f0c3cc088 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:28.054433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b41372-b825-405d-9c01-e302c7a01d8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:28.055148 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b41372-b825-405d-9c01-e302c7a01d8a admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:28.055430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b41372-b825-405d-9c01-e302c7a01d8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:28.055710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b41372-b825-405d-9c01-e302c7a01d8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:28.056435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b41372-b825-405d-9c01-e302c7a01d8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:28.058059 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b41372-b825-405d-9c01-e302c7a01d8a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:28.067990 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1343/2683] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:28.071751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594a927-a735-4e47-8fb7-922971522e56 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:28.081713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9594a927-a735-4e47-8fb7-922971522e56 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:28.086818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00b94e5c-fb55-48fd-b65c-db0614c80937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:28.087459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94e5c-fb55-48fd-b65c-db0614c80937 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:28.087737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94e5c-fb55-48fd-b65c-db0614c80937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:28.088007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00b94e5c-fb55-48fd-b65c-db0614c80937 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:28.088719 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b94e5c-fb55-48fd-b65c-db0614c80937 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:28.090324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00b94e5c-fb55-48fd-b65c-db0614c80937 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:28.099929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1341/2684] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:28 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:28.104431 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b41372-b825-405d-9c01-e302c7a01d8a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:28.114629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b41372-b825-405d-9c01-e302c7a01d8a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:28.119413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe71a3-eb9a-4c37-b890-371111cf99a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:28.120284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe71a3-eb9a-4c37-b890-371111cf99a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:28.120587 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe71a3-eb9a-4c37-b890-371111cf99a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:28.120844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbe71a3-eb9a-4c37-b890-371111cf99a5 admin admin] PATH_INFO: `/v3/projects/dfa55c7d33d2427daa4e986c7245f900/users/7269aa77fb7f45bbb12f069e67b6596f/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:28.135358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe71a3-eb9a-4c37-b890-371111cf99a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dfa55c7d33d2427daa4e986c7245f900, user_id=7269aa77fb7f45bbb12f069e67b6596f, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:28.138080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbe71a3-eb9a-4c37-b890-371111cf99a5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:28.152854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1344/2685] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:28 2022] PUT /identity/v3/projects/dfa55c7d33d2427daa4e986c7245f900/users/7269aa77fb7f45bbb12f069e67b6596f/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:28.157934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152891d5-b8ee-4b43-aa14-2477a99c8477 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:28.158225 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152891d5-b8ee-4b43-aa14-2477a99c8477 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:28.158506 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152891d5-b8ee-4b43-aa14-2477a99c8477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:28.176492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-152891d5-b8ee-4b43-aa14-2477a99c8477 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:28.177721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-152891d5-b8ee-4b43-aa14-2477a99c8477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:28.237060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1342/2686] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:28.246722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe71a3-eb9a-4c37-b890-371111cf99a5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:28.248486 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1cbe71a3-eb9a-4c37-b890-371111cf99a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:28.305682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbe71a3-eb9a-4c37-b890-371111cf99a5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:28.309392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0250-8b5d-4b31-bcd7-843007eed683 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:28.310080 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0250-8b5d-4b31-bcd7-843007eed683 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:28.310369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0250-8b5d-4b31-bcd7-843007eed683 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:28.310647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09a0250-8b5d-4b31-bcd7-843007eed683 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:28.323057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a0250-8b5d-4b31-bcd7-843007eed683 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:28.324426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09a0250-8b5d-4b31-bcd7-843007eed683 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:28.334166 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1345/2687] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:28.969771 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d241b8-396a-40f7-b3e2-5e57684b5b9f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:28.970212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d241b8-396a-40f7-b3e2-5e57684b5b9f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:28.970374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d241b8-396a-40f7-b3e2-5e57684b5b9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:29.004477 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-e9d241b8-396a-40f7-b3e2-5e57684b5b9f None None] MFA Rules not processed for user `7269aa77fb7f45bbb12f069e67b6596f`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:29.006150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-e9d241b8-396a-40f7-b3e2-5e57684b5b9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:29.030214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:29.119428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1343/2688] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:28 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:29.121362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0250-8b5d-4b31-bcd7-843007eed683 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:29.131920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09a0250-8b5d-4b31-bcd7-843007eed683 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:29.137340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bda65-d7e1-4739-bc5a-8a83cf8e19ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:29.138064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bda65-d7e1-4739-bc5a-8a83cf8e19ba admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:29.138349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bda65-d7e1-4739-bc5a-8a83cf8e19ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:29.138771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626bda65-d7e1-4739-bc5a-8a83cf8e19ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:29.139464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bda65-d7e1-4739-bc5a-8a83cf8e19ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:29.141351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626bda65-d7e1-4739-bc5a-8a83cf8e19ba admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:29.170247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1346/2689] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:29 2022] POST /identity/v3/projects => generated 405 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:29.175005 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d241b8-396a-40f7-b3e2-5e57684b5b9f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:29.187527 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d241b8-396a-40f7-b3e2-5e57684b5b9f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:29.253245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b9f99-c656-44f8-b1c0-615a63c18c63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:29.253952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b9f99-c656-44f8-b1c0-615a63c18c63 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:29.254243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b9f99-c656-44f8-b1c0-615a63c18c63 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:29.254526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898b9f99-c656-44f8-b1c0-615a63c18c63 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:29.255247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b9f99-c656-44f8-b1c0-615a63c18c63 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:29.257187 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898b9f99-c656-44f8-b1c0-615a63c18c63 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:29.283778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1344/2690] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:29 2022] POST /identity/v3/users => generated 309 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:29.283915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bda65-d7e1-4739-bc5a-8a83cf8e19ba admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:29.295402 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626bda65-d7e1-4739-bc5a-8a83cf8e19ba admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:29.301156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be142a-9893-4c8e-ac94-a91f2ebed289 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:29.302201 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be142a-9893-4c8e-ac94-a91f2ebed289 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:29.302618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be142a-9893-4c8e-ac94-a91f2ebed289 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:29.303012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44be142a-9893-4c8e-ac94-a91f2ebed289 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:29.303931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be142a-9893-4c8e-ac94-a91f2ebed289 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:29.309404 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44be142a-9893-4c8e-ac94-a91f2ebed289 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:29.321354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1347/2691] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:10:29.325671 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b9f99-c656-44f8-b1c0-615a63c18c63 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:29.335941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898b9f99-c656-44f8-b1c0-615a63c18c63 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:29.341195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfce78-10cd-47e7-8077-c5f75ffea98e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:29.342115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfce78-10cd-47e7-8077-c5f75ffea98e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:29.342407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfce78-10cd-47e7-8077-c5f75ffea98e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:29.342710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aacfce78-10cd-47e7-8077-c5f75ffea98e admin admin] PATH_INFO: `/v3/projects/fe8399fd13904304b2b963a933c69fc3/users/4eae17f7c1e847869b1e7a74eff7d654/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:29.358013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfce78-10cd-47e7-8077-c5f75ffea98e admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8399fd13904304b2b963a933c69fc3, user_id=4eae17f7c1e847869b1e7a74eff7d654, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:29.360692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aacfce78-10cd-47e7-8077-c5f75ffea98e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:29.373587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1345/2692] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:29 2022] PUT /identity/v3/projects/fe8399fd13904304b2b963a933c69fc3/users/4eae17f7c1e847869b1e7a74eff7d654/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:29.377568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be142a-9893-4c8e-ac94-a91f2ebed289 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:29.388440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44be142a-9893-4c8e-ac94-a91f2ebed289 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:29.444643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95e19b6-1357-480c-8a8b-7561e1547d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:29.445347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e19b6-1357-480c-8a8b-7561e1547d79 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:29.446172 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e19b6-1357-480c-8a8b-7561e1547d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:29.446453 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95e19b6-1357-480c-8a8b-7561e1547d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:29.447181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e19b6-1357-480c-8a8b-7561e1547d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:29.448777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95e19b6-1357-480c-8a8b-7561e1547d79 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:29.458406 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1348/2693] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:29 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:29.462606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfce78-10cd-47e7-8077-c5f75ffea98e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:29.472826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aacfce78-10cd-47e7-8077-c5f75ffea98e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:29.477537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bdb5a-952c-4465-b48b-3a58e12ca18c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:29.478421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bdb5a-952c-4465-b48b-3a58e12ca18c admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:29.478707 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bdb5a-952c-4465-b48b-3a58e12ca18c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:29.478985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-192bdb5a-952c-4465-b48b-3a58e12ca18c admin admin] PATH_INFO: `/v3/projects/fe8399fd13904304b2b963a933c69fc3/users/4eae17f7c1e847869b1e7a74eff7d654/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:29.481238 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bdb5a-952c-4465-b48b-3a58e12ca18c admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe8399fd13904304b2b963a933c69fc3, user_id=4eae17f7c1e847869b1e7a74eff7d654, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:29.483923 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-192bdb5a-952c-4465-b48b-3a58e12ca18c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:29.494763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1346/2694] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:29 2022] PUT /identity/v3/projects/fe8399fd13904304b2b963a933c69fc3/users/4eae17f7c1e847869b1e7a74eff7d654/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:30.090695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25a099-fe45-4a24-9b2f-0485615fde42 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:30.090985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25a099-fe45-4a24-9b2f-0485615fde42 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:30.091269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a25a099-fe45-4a24-9b2f-0485615fde42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:30.121850 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0a25a099-fe45-4a24-9b2f-0485615fde42 None None] MFA Rules not processed for user `4eae17f7c1e847869b1e7a74eff7d654`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:30.123094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0a25a099-fe45-4a24-9b2f-0485615fde42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:30.133490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:30.195896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1349/2695] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:30 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:31.206579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bdb5a-952c-4465-b48b-3a58e12ca18c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:31.268693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-192bdb5a-952c-4465-b48b-3a58e12ca18c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:31.273230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:31.274766 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:31.275052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:31.275322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] PATH_INFO: `/v3/users/7269aa77fb7f45bbb12f069e67b6596f` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:31.276504 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7269aa77fb7f45bbb12f069e67b6596f)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:31.278445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:31.291747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:31.332428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7269aa77fb7f45bbb12f069e67b6596f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.345601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7269aa77fb7f45bbb12f069e67b6596f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.353151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7269aa77fb7f45bbb12f069e67b6596f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.353883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7269aa77fb7f45bbb12f069e67b6596f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.354484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7269aa77fb7f45bbb12f069e67b6596f'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.370586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1347/2696] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:10:31 2022] DELETE /identity/v3/users/7269aa77fb7f45bbb12f069e67b6596f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:31.537681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25a099-fe45-4a24-9b2f-0485615fde42 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:31.548818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a25a099-fe45-4a24-9b2f-0485615fde42 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:31.608024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:31.608891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:31.609177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:31.609458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] PATH_INFO: `/v3/projects/dfa55c7d33d2427daa4e986c7245f900` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:31.610678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfa55c7d33d2427daa4e986c7245f900)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:31.612593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:31.658629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfa55c7d33d2427daa4e986c7245f900', 'request_id': 'req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.665708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfa55c7d33d2427daa4e986c7245f900', 'request_id': 'req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.671292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfa55c7d33d2427daa4e986c7245f900', 'request_id': 'req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.683763 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] The token cache is being invalidate because project dfa55c7d33d2427daa4e986c7245f900 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:10:31.684071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.684738 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.686538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1350/2697] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:10:31 2022] DELETE /identity/v3/projects/dfa55c7d33d2427daa4e986c7245f900 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:31.690721 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:31.692113 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:31.705408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0102284b-1b22-456e-b55d-ed6d66047732 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:31.760691 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:31.761586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:31.761874 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:31.762151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] PATH_INFO: `/v3/users/4eae17f7c1e847869b1e7a74eff7d654` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:31.763337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4eae17f7c1e847869b1e7a74eff7d654)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:31.765257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:31.811202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4eae17f7c1e847869b1e7a74eff7d654'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.822848 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4eae17f7c1e847869b1e7a74eff7d654'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.829652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4eae17f7c1e847869b1e7a74eff7d654'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.830316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4eae17f7c1e847869b1e7a74eff7d654'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.830918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-05653c1c-4f90-4a05-b151-610b140e006b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4eae17f7c1e847869b1e7a74eff7d654'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:31.843798 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1348/2698] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:10:31 2022] DELETE /identity/v3/users/4eae17f7c1e847869b1e7a74eff7d654 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:32.014762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.016183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:32.031533 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f2efec1-bf34-4236-b5e9-8e5c04974d1e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.066079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:32.091064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.091919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.092199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.092469 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] PATH_INFO: `/v3/projects/fe8399fd13904304b2b963a933c69fc3` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.093659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe8399fd13904304b2b963a933c69fc3)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.095552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.140925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe8399fd13904304b2b963a933c69fc3', 'request_id': 'req-5e777573-aba9-404f-8483-31c9d9618a28'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:32.147231 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe8399fd13904304b2b963a933c69fc3', 'request_id': 'req-5e777573-aba9-404f-8483-31c9d9618a28'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:32.153662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe8399fd13904304b2b963a933c69fc3', 'request_id': 'req-5e777573-aba9-404f-8483-31c9d9618a28'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:32.165951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] The token cache is being invalidate because project fe8399fd13904304b2b963a933c69fc3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:10:32.166234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:32.166841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-5e777573-aba9-404f-8483-31c9d9618a28 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:32.168345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1351/2699] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:10:32 2022] DELETE /identity/v3/projects/fe8399fd13904304b2b963a933c69fc3 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:32.173309 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c283b1-7a57-47f3-8d28-31ebfc1d1955 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.173644 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c283b1-7a57-47f3-8d28-31ebfc1d1955 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.173932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c283b1-7a57-47f3-8d28-31ebfc1d1955 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.191333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-12c283b1-7a57-47f3-8d28-31ebfc1d1955 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:32.192576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-12c283b1-7a57-47f3-8d28-31ebfc1d1955 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:32.197294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:32.254775 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1349/2700] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:32.263152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ba512-102e-4500-b5ce-c8ee74c4a13c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.263443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ba512-102e-4500-b5ce-c8ee74c4a13c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.263748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a4ba512-102e-4500-b5ce-c8ee74c4a13c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.282706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-8a4ba512-102e-4500-b5ce-c8ee74c4a13c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:32.283911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-8a4ba512-102e-4500-b5ce-c8ee74c4a13c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:32.291976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1352/2701] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:32.296564 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c283b1-7a57-47f3-8d28-31ebfc1d1955 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.306935 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c283b1-7a57-47f3-8d28-31ebfc1d1955 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.311808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108347d3-ea1d-4f33-b7b3-4d2153ae898b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.312473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108347d3-ea1d-4f33-b7b3-4d2153ae898b admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.312761 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108347d3-ea1d-4f33-b7b3-4d2153ae898b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.313039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108347d3-ea1d-4f33-b7b3-4d2153ae898b admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.314472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108347d3-ea1d-4f33-b7b3-4d2153ae898b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:10:32.314788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108347d3-ea1d-4f33-b7b3-4d2153ae898b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.316422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108347d3-ea1d-4f33-b7b3-4d2153ae898b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.329837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1350/2702] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:10:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:10:32.334145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ba512-102e-4500-b5ce-c8ee74c4a13c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.344484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a4ba512-102e-4500-b5ce-c8ee74c4a13c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.349123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa751ea4-9648-4c59-8de0-9e1b95bc46af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.349806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa751ea4-9648-4c59-8de0-9e1b95bc46af admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.350087 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa751ea4-9648-4c59-8de0-9e1b95bc46af admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.350361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa751ea4-9648-4c59-8de0-9e1b95bc46af admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.351053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa751ea4-9648-4c59-8de0-9e1b95bc46af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.353584 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa751ea4-9648-4c59-8de0-9e1b95bc46af admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.386091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1353/2703] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:32 2022] POST /identity/v3/projects => generated 365 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:32.387261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108347d3-ea1d-4f33-b7b3-4d2153ae898b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.397464 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108347d3-ea1d-4f33-b7b3-4d2153ae898b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.454137 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c47a6-b51d-4612-9a8b-f619c2ddc448 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.454801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c47a6-b51d-4612-9a8b-f619c2ddc448 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.455087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c47a6-b51d-4612-9a8b-f619c2ddc448 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.455363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c47a6-b51d-4612-9a8b-f619c2ddc448 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.456075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c47a6-b51d-4612-9a8b-f619c2ddc448 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.458074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058c47a6-b51d-4612-9a8b-f619c2ddc448 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.477963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1351/2704] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:32 2022] POST /identity/v3/users => generated 283 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:32.482504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa751ea4-9648-4c59-8de0-9e1b95bc46af admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.493305 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa751ea4-9648-4c59-8de0-9e1b95bc46af admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.497969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2172fe-33f8-417d-b3d0-6957edd3aef7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.498620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2172fe-33f8-417d-b3d0-6957edd3aef7 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.498906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2172fe-33f8-417d-b3d0-6957edd3aef7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.499184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2172fe-33f8-417d-b3d0-6957edd3aef7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.499910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2172fe-33f8-417d-b3d0-6957edd3aef7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.501498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2172fe-33f8-417d-b3d0-6957edd3aef7 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.511915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1354/2705] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:32.515838 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c47a6-b51d-4612-9a8b-f619c2ddc448 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.526471 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058c47a6-b51d-4612-9a8b-f619c2ddc448 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.531067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17392959-4882-44da-982c-4de573e6c8b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.531740 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17392959-4882-44da-982c-4de573e6c8b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.532017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17392959-4882-44da-982c-4de573e6c8b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.532291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17392959-4882-44da-982c-4de573e6c8b9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.533005 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17392959-4882-44da-982c-4de573e6c8b9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.534610 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17392959-4882-44da-982c-4de573e6c8b9 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.545114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1352/2706] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:32.549053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2172fe-33f8-417d-b3d0-6957edd3aef7 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.559026 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2172fe-33f8-417d-b3d0-6957edd3aef7 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.563577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bcb80-60c0-46a4-a868-412843458b76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.564439 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bcb80-60c0-46a4-a868-412843458b76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.564718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bcb80-60c0-46a4-a868-412843458b76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.564989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354bcb80-60c0-46a4-a868-412843458b76 admin admin] PATH_INFO: `/v3/projects/1271a6640def45b4adff9981438e517f/users/20b25d254cc6464dbaeea16645afcc8d/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.579164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bcb80-60c0-46a4-a868-412843458b76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1271a6640def45b4adff9981438e517f, user_id=20b25d254cc6464dbaeea16645afcc8d, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.581803 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354bcb80-60c0-46a4-a868-412843458b76 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.594889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1355/2707] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:32 2022] PUT /identity/v3/projects/1271a6640def45b4adff9981438e517f/users/20b25d254cc6464dbaeea16645afcc8d/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:32.606928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb5205-ac32-491e-a1a5-313f5aafc5f2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.607219 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb5205-ac32-491e-a1a5-313f5aafc5f2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.607500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48cb5205-ac32-491e-a1a5-313f5aafc5f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.633281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-48cb5205-ac32-491e-a1a5-313f5aafc5f2 None None] MFA Rules not processed for user `20b25d254cc6464dbaeea16645afcc8d`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:32.634523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-48cb5205-ac32-491e-a1a5-313f5aafc5f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:32.696290 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1353/2708] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:32.700658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bcb80-60c0-46a4-a868-412843458b76 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.762106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354bcb80-60c0-46a4-a868-412843458b76 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.766847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dd6cd-8992-47c2-8542-b5b3146efb80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.767492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dd6cd-8992-47c2-8542-b5b3146efb80 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.767775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dd6cd-8992-47c2-8542-b5b3146efb80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.768050 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88dd6cd-8992-47c2-8542-b5b3146efb80 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.768739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88dd6cd-8992-47c2-8542-b5b3146efb80 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.770675 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b88dd6cd-8992-47c2-8542-b5b3146efb80 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.779024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:32.797953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1356/2709] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:32 2022] POST /identity/v3/projects => generated 363 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:10:32.802033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb5205-ac32-491e-a1a5-313f5aafc5f2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.808847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:32.813886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48cb5205-ac32-491e-a1a5-313f5aafc5f2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.870967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35874b-c7f6-49bc-b347-8384f29e421e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.871646 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35874b-c7f6-49bc-b347-8384f29e421e admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.871932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35874b-c7f6-49bc-b347-8384f29e421e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.872209 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35874b-c7f6-49bc-b347-8384f29e421e admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.872900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35874b-c7f6-49bc-b347-8384f29e421e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.874845 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35874b-c7f6-49bc-b347-8384f29e421e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.894556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1354/2710] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:32 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:32.899271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dd6cd-8992-47c2-8542-b5b3146efb80 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.909756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88dd6cd-8992-47c2-8542-b5b3146efb80 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.914677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c67c50-fe5c-4ae2-98c2-81bcca44987b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.915362 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c67c50-fe5c-4ae2-98c2-81bcca44987b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.915650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c67c50-fe5c-4ae2-98c2-81bcca44987b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.915928 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c67c50-fe5c-4ae2-98c2-81bcca44987b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.916650 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c67c50-fe5c-4ae2-98c2-81bcca44987b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.918377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c67c50-fe5c-4ae2-98c2-81bcca44987b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.929132 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1357/2711] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:10:32.933463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35874b-c7f6-49bc-b347-8384f29e421e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.943484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35874b-c7f6-49bc-b347-8384f29e421e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:32.948853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcdd3c-81e8-4e64-aef5-f828decb4841 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:32.949730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcdd3c-81e8-4e64-aef5-f828decb4841 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:32.950017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcdd3c-81e8-4e64-aef5-f828decb4841 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:32.950295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fcdd3c-81e8-4e64-aef5-f828decb4841 admin admin] PATH_INFO: `/v3/projects/1bbcb631a726487b90029ba7989881c7/users/be782a88bf504a419ea3170756ad1e92/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:32.965563 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcdd3c-81e8-4e64-aef5-f828decb4841 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbcb631a726487b90029ba7989881c7, user_id=be782a88bf504a419ea3170756ad1e92, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:32.968570 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fcdd3c-81e8-4e64-aef5-f828decb4841 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:32.981565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1355/2712] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:32 2022] PUT /identity/v3/projects/1bbcb631a726487b90029ba7989881c7/users/be782a88bf504a419ea3170756ad1e92/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:10:32.985446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c67c50-fe5c-4ae2-98c2-81bcca44987b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:32.995705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c67c50-fe5c-4ae2-98c2-81bcca44987b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:33.051138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6545a28-aaa1-4a7b-bd43-945a9e434b0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:33.051798 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6545a28-aaa1-4a7b-bd43-945a9e434b0d admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:33.052082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6545a28-aaa1-4a7b-bd43-945a9e434b0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:33.052361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6545a28-aaa1-4a7b-bd43-945a9e434b0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:33.053078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6545a28-aaa1-4a7b-bd43-945a9e434b0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:33.054712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6545a28-aaa1-4a7b-bd43-945a9e434b0d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:33.064385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1358/2713] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:32 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:33.068423 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcdd3c-81e8-4e64-aef5-f828decb4841 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:33.079294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fcdd3c-81e8-4e64-aef5-f828decb4841 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:33.083926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94654bc-e73f-4675-b25d-7a6b2095dfb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:33.084801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94654bc-e73f-4675-b25d-7a6b2095dfb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:33.085084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94654bc-e73f-4675-b25d-7a6b2095dfb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:33.085361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94654bc-e73f-4675-b25d-7a6b2095dfb4 admin admin] PATH_INFO: `/v3/projects/1bbcb631a726487b90029ba7989881c7/users/be782a88bf504a419ea3170756ad1e92/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:33.088063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94654bc-e73f-4675-b25d-7a6b2095dfb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1bbcb631a726487b90029ba7989881c7, user_id=be782a88bf504a419ea3170756ad1e92, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:33.090776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94654bc-e73f-4675-b25d-7a6b2095dfb4 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:33.102104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1356/2714] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:33 2022] PUT /identity/v3/projects/1bbcb631a726487b90029ba7989881c7/users/be782a88bf504a419ea3170756ad1e92/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:33.112300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331290a8-c2f9-49f4-98d9-9fc45ce74e50 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:33.112590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331290a8-c2f9-49f4-98d9-9fc45ce74e50 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:33.112875 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331290a8-c2f9-49f4-98d9-9fc45ce74e50 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:33.140113 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-331290a8-c2f9-49f4-98d9-9fc45ce74e50 None None] MFA Rules not processed for user `be782a88bf504a419ea3170756ad1e92`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:33.141310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-331290a8-c2f9-49f4-98d9-9fc45ce74e50 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:33.210874 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1359/2715] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:33 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:33.222014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94654bc-e73f-4675-b25d-7a6b2095dfb4 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:33.223757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c94654bc-e73f-4675-b25d-7a6b2095dfb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:33.280407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94654bc-e73f-4675-b25d-7a6b2095dfb4 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:33.284004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd337312-319f-484c-9f60-c95ff30cc796 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:33.284635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd337312-319f-484c-9f60-c95ff30cc796 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:33.284916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd337312-319f-484c-9f60-c95ff30cc796 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:33.285191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd337312-319f-484c-9f60-c95ff30cc796 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:33.294134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd337312-319f-484c-9f60-c95ff30cc796 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:33.295566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd337312-319f-484c-9f60-c95ff30cc796 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:33.305377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1357/2716] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:33 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:33.382201 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331290a8-c2f9-49f4-98d9-9fc45ce74e50 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:33.393245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331290a8-c2f9-49f4-98d9-9fc45ce74e50 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:33.396919 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ca655-0cfc-4bc4-b58c-032722b588b0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:33.397595 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca655-0cfc-4bc4-b58c-032722b588b0 service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:33.397885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca655-0cfc-4bc4-b58c-032722b588b0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:33.398156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ca655-0cfc-4bc4-b58c-032722b588b0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:33.455825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ca655-0cfc-4bc4-b58c-032722b588b0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:33.457266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ca655-0cfc-4bc4-b58c-032722b588b0 service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:33.460816 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:33.467283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1360/2717] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:43.300695 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=68,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:43.469982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:50.035317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd337312-319f-484c-9f60-c95ff30cc796 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:50.114043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd337312-319f-484c-9f60-c95ff30cc796 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:50.118693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:50.119654 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:50.119962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:50.120235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] PATH_INFO: `/v3/users/20b25d254cc6464dbaeea16645afcc8d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:50.121422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] RBAC: Authorizing `identity:delete_user(user_id=20b25d254cc6464dbaeea16645afcc8d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:50.123475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:50.170832 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '20b25d254cc6464dbaeea16645afcc8d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.182560 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '20b25d254cc6464dbaeea16645afcc8d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.189132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '20b25d254cc6464dbaeea16645afcc8d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.189824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '20b25d254cc6464dbaeea16645afcc8d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.190445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '20b25d254cc6464dbaeea16645afcc8d'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.203742 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1358/2718] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:10:50 2022] DELETE /identity/v3/users/20b25d254cc6464dbaeea16645afcc8d => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:50.208368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ea2ce-7f32-4a89-9a36-bc9bc78c314e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:50.208821 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ea2ce-7f32-4a89-9a36-bc9bc78c314e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:50.208940 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-637ea2ce-7f32-4a89-9a36-bc9bc78c314e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:50.227073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-637ea2ce-7f32-4a89-9a36-bc9bc78c314e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:50.228272 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-637ea2ce-7f32-4a89-9a36-bc9bc78c314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:50.285366 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1361/2719] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:50.294984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:50.296382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:50.355075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b3ce722-a64e-439c-89d4-0a3a517bef5b admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:50.359072 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9e9a4-7a3c-4eec-8b7a-8f236f8f55a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:50.359659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9e9a4-7a3c-4eec-8b7a-8f236f8f55a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:50.359943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9e9a4-7a3c-4eec-8b7a-8f236f8f55a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:50.360220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a9e9a4-7a3c-4eec-8b7a-8f236f8f55a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:50.369075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9e9a4-7a3c-4eec-8b7a-8f236f8f55a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:50.370629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a9e9a4-7a3c-4eec-8b7a-8f236f8f55a2 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:50.380001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1359/2720] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:50.646526 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ea2ce-7f32-4a89-9a36-bc9bc78c314e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:50.647981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-637ea2ce-7f32-4a89-9a36-bc9bc78c314e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:50.661131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-637ea2ce-7f32-4a89-9a36-bc9bc78c314e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:50.665668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:50.666656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:50.666946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:50.667222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] PATH_INFO: `/v3/projects/1271a6640def45b4adff9981438e517f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:50.668400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=1271a6640def45b4adff9981438e517f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:50.670325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:50.717702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1271a6640def45b4adff9981438e517f', 'request_id': 'req-3ac81072-f0b2-45d2-a786-14d6b9376c2f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.723822 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1271a6640def45b4adff9981438e517f', 'request_id': 'req-3ac81072-f0b2-45d2-a786-14d6b9376c2f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.729481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1271a6640def45b4adff9981438e517f', 'request_id': 'req-3ac81072-f0b2-45d2-a786-14d6b9376c2f'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.741781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] The token cache is being invalidate because project 1271a6640def45b4adff9981438e517f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:10:50.742069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.742729 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.744773 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1362/2721] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:10:50 2022] DELETE /identity/v3/projects/1271a6640def45b4adff9981438e517f => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:50.748814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9e9a4-7a3c-4eec-8b7a-8f236f8f55a2 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:50.758432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a9e9a4-7a3c-4eec-8b7a-8f236f8f55a2 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:50.776943 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:50.816439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:50.817304 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:50.817620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:50.817911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] PATH_INFO: `/v3/users/be782a88bf504a419ea3170756ad1e92` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:50.819065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] RBAC: Authorizing `identity:delete_user(user_id=be782a88bf504a419ea3170756ad1e92)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:50.820980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:50.865240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be782a88bf504a419ea3170756ad1e92'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.877251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be782a88bf504a419ea3170756ad1e92'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.883895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be782a88bf504a419ea3170756ad1e92'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.884634 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be782a88bf504a419ea3170756ad1e92'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.885352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-61cccc6c-609e-493a-bbe8-5bccec8cb89f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be782a88bf504a419ea3170756ad1e92'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:50.897547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1360/2722] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:10:50 2022] DELETE /identity/v3/users/be782a88bf504a419ea3170756ad1e92 => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:51.160571 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:51.162198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:51.176063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac81072-f0b2-45d2-a786-14d6b9376c2f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:51.248208 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:51.249160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:51.249528 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:51.249895 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] PATH_INFO: `/v3/projects/1bbcb631a726487b90029ba7989881c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:51.251226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1bbcb631a726487b90029ba7989881c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:51.253361 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:51.286839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:51.328980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1bbcb631a726487b90029ba7989881c7', 'request_id': 'req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:51.335354 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1bbcb631a726487b90029ba7989881c7', 'request_id': 'req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:51.342339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1bbcb631a726487b90029ba7989881c7', 'request_id': 'req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:51.355521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] The token cache is being invalidate because project 1bbcb631a726487b90029ba7989881c7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:10:51.355809 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:51.356444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-079ee3dd-91c7-4b7e-8e44-0b0991f71c52 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:51.358282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1363/2723] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:10:51 2022] DELETE /identity/v3/projects/1bbcb631a726487b90029ba7989881c7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:51.363473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d46307-6288-41aa-8a4e-f5e868579744 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:51.363948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d46307-6288-41aa-8a4e-f5e868579744 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:51.364099 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d46307-6288-41aa-8a4e-f5e868579744 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:51.384452 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-66d46307-6288-41aa-8a4e-f5e868579744 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:51.389117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-66d46307-6288-41aa-8a4e-f5e868579744 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:51.444710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1361/2724] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:51.453618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbbcbb-ad4c-4015-a1d0-7c170bad237e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:51.453909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbbcbb-ad4c-4015-a1d0-7c170bad237e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:51.454185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fbbcbb-ad4c-4015-a1d0-7c170bad237e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:51.472670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-e3fbbcbb-ad4c-4015-a1d0-7c170bad237e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:51.473900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e3fbbcbb-ad4c-4015-a1d0-7c170bad237e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:51.482745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1364/2725] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:10:51.486922 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d46307-6288-41aa-8a4e-f5e868579744 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:51.496665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d46307-6288-41aa-8a4e-f5e868579744 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:51.501260 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97a8b6-e137-41c6-b16b-27d2561e1c69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:51.501946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97a8b6-e137-41c6-b16b-27d2561e1c69 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:51.502233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97a8b6-e137-41c6-b16b-27d2561e1c69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:51.502512 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d97a8b6-e137-41c6-b16b-27d2561e1c69 admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:51.503231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97a8b6-e137-41c6-b16b-27d2561e1c69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:10:51.503537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97a8b6-e137-41c6-b16b-27d2561e1c69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:51.505134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d97a8b6-e137-41c6-b16b-27d2561e1c69 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:51.517059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1362/2726] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:10:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 20:10:51.521255 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbbcbb-ad4c-4015-a1d0-7c170bad237e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:51.531000 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fbbcbb-ad4c-4015-a1d0-7c170bad237e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:51.535629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45784f2-080c-4827-878e-77deb63ed9a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:51.536266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45784f2-080c-4827-878e-77deb63ed9a8 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:51.536552 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45784f2-080c-4827-878e-77deb63ed9a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:51.536828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c45784f2-080c-4827-878e-77deb63ed9a8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:51.537539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45784f2-080c-4827-878e-77deb63ed9a8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:51.539430 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c45784f2-080c-4827-878e-77deb63ed9a8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:51.563703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1365/2727] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:51 2022] POST /identity/v3/projects => generated 369 bytes in 43 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:51.567778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97a8b6-e137-41c6-b16b-27d2561e1c69 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:51.578366 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d97a8b6-e137-41c6-b16b-27d2561e1c69 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:51.603699 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:51.636230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a69cd-0501-4d28-94fc-dc83d565fb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:51.636892 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a69cd-0501-4d28-94fc-dc83d565fb6e admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:51.637178 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a69cd-0501-4d28-94fc-dc83d565fb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:51.637448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050a69cd-0501-4d28-94fc-dc83d565fb6e admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:51.638167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a69cd-0501-4d28-94fc-dc83d565fb6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:51.640060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050a69cd-0501-4d28-94fc-dc83d565fb6e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:51.659369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1363/2728] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:51 2022] POST /identity/v3/users => generated 285 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:10:51.663748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45784f2-080c-4827-878e-77deb63ed9a8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:51.676674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c45784f2-080c-4827-878e-77deb63ed9a8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:51.681708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75df66a-099a-4f91-b01a-8c16a9030458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:51.682489 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75df66a-099a-4f91-b01a-8c16a9030458 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:51.682877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75df66a-099a-4f91-b01a-8c16a9030458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:51.683255 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75df66a-099a-4f91-b01a-8c16a9030458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:51.684353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75df66a-099a-4f91-b01a-8c16a9030458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:51.686797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75df66a-099a-4f91-b01a-8c16a9030458 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:51.699883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1366/2729] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:51.703718 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a69cd-0501-4d28-94fc-dc83d565fb6e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:51.715162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050a69cd-0501-4d28-94fc-dc83d565fb6e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:51.720019 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f2fa0e3-bb7c-4755-82ed-bfd888f95b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:51.720680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fa0e3-bb7c-4755-82ed-bfd888f95b6c admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:51.720966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fa0e3-bb7c-4755-82ed-bfd888f95b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:51.721246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f2fa0e3-bb7c-4755-82ed-bfd888f95b6c admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:51.722039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fa0e3-bb7c-4755-82ed-bfd888f95b6c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:51.723631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f2fa0e3-bb7c-4755-82ed-bfd888f95b6c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:51.733955 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1364/2730] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:51.738108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75df66a-099a-4f91-b01a-8c16a9030458 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:51.748138 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75df66a-099a-4f91-b01a-8c16a9030458 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:51.752854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e4e86-e150-4962-957c-0bd86f3b594a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:51.753753 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e4e86-e150-4962-957c-0bd86f3b594a admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:51.754039 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e4e86-e150-4962-957c-0bd86f3b594a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:51.754322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac4e4e86-e150-4962-957c-0bd86f3b594a admin admin] PATH_INFO: `/v3/projects/f73bd8a9a77c4f36a2a2b36e35e049ef/users/4d0d7bc3d2bc4802ac779779dde8dd7e/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:51.769053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e4e86-e150-4962-957c-0bd86f3b594a admin admin] RBAC: Authorizing `identity:create_grant(project_id=f73bd8a9a77c4f36a2a2b36e35e049ef, user_id=4d0d7bc3d2bc4802ac779779dde8dd7e, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:51.771744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac4e4e86-e150-4962-957c-0bd86f3b594a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:51.784967 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1367/2731] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:51 2022] PUT /identity/v3/projects/f73bd8a9a77c4f36a2a2b36e35e049ef/users/4d0d7bc3d2bc4802ac779779dde8dd7e/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:51.844970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67803995-6942-4384-bc25-31b055e305fc None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:51.845256 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67803995-6942-4384-bc25-31b055e305fc None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:51.845561 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67803995-6942-4384-bc25-31b055e305fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:51.873117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-67803995-6942-4384-bc25-31b055e305fc None None] MFA Rules not processed for user `4d0d7bc3d2bc4802ac779779dde8dd7e`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:51.874339 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-67803995-6942-4384-bc25-31b055e305fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:51.936111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1365/2732] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:51 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:51.940613 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e4e86-e150-4962-957c-0bd86f3b594a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:52.005562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac4e4e86-e150-4962-957c-0bd86f3b594a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:52.010578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5001b9c-3fac-41bc-9f1d-d8dfa81db357 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:52.011228 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5001b9c-3fac-41bc-9f1d-d8dfa81db357 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:52.011515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5001b9c-3fac-41bc-9f1d-d8dfa81db357 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:52.011793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5001b9c-3fac-41bc-9f1d-d8dfa81db357 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:52.012488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5001b9c-3fac-41bc-9f1d-d8dfa81db357 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:52.014458 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5001b9c-3fac-41bc-9f1d-d8dfa81db357 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:52.039563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1368/2733] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:51 2022] POST /identity/v3/projects => generated 369 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:52.044327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67803995-6942-4384-bc25-31b055e305fc None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:52.055030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67803995-6942-4384-bc25-31b055e305fc None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:52.111142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562a054-cf6c-4cdd-8df4-30d3a5a257bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:52.111815 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562a054-cf6c-4cdd-8df4-30d3a5a257bb admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:52.112101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562a054-cf6c-4cdd-8df4-30d3a5a257bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:52.112381 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562a054-cf6c-4cdd-8df4-30d3a5a257bb admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:52.113073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562a054-cf6c-4cdd-8df4-30d3a5a257bb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:52.115030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562a054-cf6c-4cdd-8df4-30d3a5a257bb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:52.134456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1366/2734] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:52 2022] POST /identity/v3/users => generated 291 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:52.138567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5001b9c-3fac-41bc-9f1d-d8dfa81db357 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:52.148844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5001b9c-3fac-41bc-9f1d-d8dfa81db357 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:52.154002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ad50-f801-4906-8bac-7097e2b3635c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:52.154648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ad50-f801-4906-8bac-7097e2b3635c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:52.154931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ad50-f801-4906-8bac-7097e2b3635c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:52.155206 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ad50-f801-4906-8bac-7097e2b3635c admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:52.155948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5ad50-f801-4906-8bac-7097e2b3635c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:52.157593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5ad50-f801-4906-8bac-7097e2b3635c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:52.168736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1369/2735] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:52.172814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562a054-cf6c-4cdd-8df4-30d3a5a257bb admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:52.183114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562a054-cf6c-4cdd-8df4-30d3a5a257bb admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:52.187778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c1405-3bb8-471d-a425-94a8f3d25f9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:52.188703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c1405-3bb8-471d-a425-94a8f3d25f9c admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:52.188984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c1405-3bb8-471d-a425-94a8f3d25f9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:52.189262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f6c1405-3bb8-471d-a425-94a8f3d25f9c admin admin] PATH_INFO: `/v3/projects/57abcbe1cc6049d0b04e073ecd52c098/users/5e9afc49c36344dcb1428c6955f8b3cd/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:52.204021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c1405-3bb8-471d-a425-94a8f3d25f9c admin admin] RBAC: Authorizing `identity:create_grant(project_id=57abcbe1cc6049d0b04e073ecd52c098, user_id=5e9afc49c36344dcb1428c6955f8b3cd, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:52.206713 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f6c1405-3bb8-471d-a425-94a8f3d25f9c admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:52.221324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1367/2736] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:52 2022] PUT /identity/v3/projects/57abcbe1cc6049d0b04e073ecd52c098/users/5e9afc49c36344dcb1428c6955f8b3cd/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:52.225313 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ad50-f801-4906-8bac-7097e2b3635c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:52.236325 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ad50-f801-4906-8bac-7097e2b3635c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:52.256275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:52.295775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57e252ae-1e49-4bdc-99a8-88bcefebdedd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:52.296440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e252ae-1e49-4bdc-99a8-88bcefebdedd admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:52.296725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e252ae-1e49-4bdc-99a8-88bcefebdedd admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:52.297004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57e252ae-1e49-4bdc-99a8-88bcefebdedd admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:52.297752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e252ae-1e49-4bdc-99a8-88bcefebdedd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:52.299331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57e252ae-1e49-4bdc-99a8-88bcefebdedd admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:52.309289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1370/2737] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:52 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:52.313825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c1405-3bb8-471d-a425-94a8f3d25f9c admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:52.319753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:52.325842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f6c1405-3bb8-471d-a425-94a8f3d25f9c admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:52.331471 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc81795-c211-45da-904a-0603e105869f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:52.332388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc81795-c211-45da-904a-0603e105869f admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:52.332670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc81795-c211-45da-904a-0603e105869f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:52.332971 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc81795-c211-45da-904a-0603e105869f admin admin] PATH_INFO: `/v3/projects/57abcbe1cc6049d0b04e073ecd52c098/users/5e9afc49c36344dcb1428c6955f8b3cd/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:52.334967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc81795-c211-45da-904a-0603e105869f admin admin] RBAC: Authorizing `identity:create_grant(project_id=57abcbe1cc6049d0b04e073ecd52c098, user_id=5e9afc49c36344dcb1428c6955f8b3cd, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:52.337658 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc81795-c211-45da-904a-0603e105869f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:52.348513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1368/2738] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:52 2022] PUT /identity/v3/projects/57abcbe1cc6049d0b04e073ecd52c098/users/5e9afc49c36344dcb1428c6955f8b3cd/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:10:52.359761 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51becc32-3b41-485c-a7ae-f370dd074735 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:52.360076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51becc32-3b41-485c-a7ae-f370dd074735 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:52.360358 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51becc32-3b41-485c-a7ae-f370dd074735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:52.387023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-51becc32-3b41-485c-a7ae-f370dd074735 None None] MFA Rules not processed for user `5e9afc49c36344dcb1428c6955f8b3cd`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:52.388234 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-51becc32-3b41-485c-a7ae-f370dd074735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:52.456597 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1371/2739] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:52 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:52.468052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc81795-c211-45da-904a-0603e105869f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:52.469837 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ebc81795-c211-45da-904a-0603e105869f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:52.527230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc81795-c211-45da-904a-0603e105869f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:52.530778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a97f7b-f899-42d0-b90f-6ef9e2b335bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:52.531415 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a97f7b-f899-42d0-b90f-6ef9e2b335bc service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:52.531703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a97f7b-f899-42d0-b90f-6ef9e2b335bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:52.531980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a97f7b-f899-42d0-b90f-6ef9e2b335bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:52.541819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a97f7b-f899-42d0-b90f-6ef9e2b335bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:52.543246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a97f7b-f899-42d0-b90f-6ef9e2b335bc service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:52.552774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1369/2740] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:52 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:53.855660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51becc32-3b41-485c-a7ae-f370dd074735 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:53.915639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51becc32-3b41-485c-a7ae-f370dd074735 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:53.920793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:53.921685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:53.921964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:53.922240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] PATH_INFO: `/v3/users/4d0d7bc3d2bc4802ac779779dde8dd7e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:53.923403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4d0d7bc3d2bc4802ac779779dde8dd7e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:53.925301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:53.969727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4d0d7bc3d2bc4802ac779779dde8dd7e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:53.970865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:53.984198 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4d0d7bc3d2bc4802ac779779dde8dd7e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:53.991046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4d0d7bc3d2bc4802ac779779dde8dd7e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:53.991721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4d0d7bc3d2bc4802ac779779dde8dd7e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:53.992327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4d0d7bc3d2bc4802ac779779dde8dd7e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.005213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1372/2741] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:10:53 2022] DELETE /identity/v3/users/4d0d7bc3d2bc4802ac779779dde8dd7e => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:54.009755 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf3a3-c572-4752-956e-6dfe3443e81f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:54.010251 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf3a3-c572-4752-956e-6dfe3443e81f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:54.010429 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbf3a3-c572-4752-956e-6dfe3443e81f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:54.027776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-92dbf3a3-c572-4752-956e-6dfe3443e81f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:54.028973 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-92dbf3a3-c572-4752-956e-6dfe3443e81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:54.085885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1370/2742] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:54.095333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:54.096756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:54.154806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c87c59-16e2-4aea-8fa3-de95b217b321 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:54.158550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c20479-ef33-43e9-a357-e08dfb76c9a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:54.159190 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c20479-ef33-43e9-a357-e08dfb76c9a6 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:54.159477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c20479-ef33-43e9-a357-e08dfb76c9a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:54.159756 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c20479-ef33-43e9-a357-e08dfb76c9a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:54.169473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c20479-ef33-43e9-a357-e08dfb76c9a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:54.170932 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c20479-ef33-43e9-a357-e08dfb76c9a6 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:54.180413 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1373/2743] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:54.452133 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbf3a3-c572-4752-956e-6dfe3443e81f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:54.454148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-92dbf3a3-c572-4752-956e-6dfe3443e81f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:54.468300 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbf3a3-c572-4752-956e-6dfe3443e81f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:54.472960 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:54.474299 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:54.474299 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:54.474584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] PATH_INFO: `/v3/projects/f73bd8a9a77c4f36a2a2b36e35e049ef` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:54.475737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f73bd8a9a77c4f36a2a2b36e35e049ef)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:54.477722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:54.526563 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f73bd8a9a77c4f36a2a2b36e35e049ef', 'request_id': 'req-6c151e0d-0709-4270-86fe-041708b77f10'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.531871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f73bd8a9a77c4f36a2a2b36e35e049ef', 'request_id': 'req-6c151e0d-0709-4270-86fe-041708b77f10'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.538954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f73bd8a9a77c4f36a2a2b36e35e049ef', 'request_id': 'req-6c151e0d-0709-4270-86fe-041708b77f10'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.550264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] The token cache is being invalidate because project f73bd8a9a77c4f36a2a2b36e35e049ef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:10:54.550576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.551234 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.552791 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1371/2744] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:10:54 2022] DELETE /identity/v3/projects/f73bd8a9a77c4f36a2a2b36e35e049ef => generated 0 bytes in 101 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:54.556786 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c20479-ef33-43e9-a357-e08dfb76c9a6 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:54.566673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c20479-ef33-43e9-a357-e08dfb76c9a6 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:54.623061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:54.623956 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:54.624624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:54.624624 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] PATH_INFO: `/v3/users/5e9afc49c36344dcb1428c6955f8b3cd` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:54.626391 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e9afc49c36344dcb1428c6955f8b3cd)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:54.628662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:54.677013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e9afc49c36344dcb1428c6955f8b3cd'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.689724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e9afc49c36344dcb1428c6955f8b3cd'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.694148 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:54.697965 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e9afc49c36344dcb1428c6955f8b3cd'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.698723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e9afc49c36344dcb1428c6955f8b3cd'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.699873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-7047fc27-591a-47c3-bfb0-9ddc33f65fa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e9afc49c36344dcb1428c6955f8b3cd'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:54.712324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1374/2745] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:10:54 2022] DELETE /identity/v3/users/5e9afc49c36344dcb1428c6955f8b3cd => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:54.976879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:54.978505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:54.992356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c151e0d-0709-4270-86fe-041708b77f10 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.004195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:55.051068 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.051940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.052250 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.052533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] PATH_INFO: `/v3/projects/57abcbe1cc6049d0b04e073ecd52c098` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.053756 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] RBAC: Authorizing `identity:delete_project(project_id=57abcbe1cc6049d0b04e073ecd52c098)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.055724 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.101916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57abcbe1cc6049d0b04e073ecd52c098', 'request_id': 'req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:55.108192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57abcbe1cc6049d0b04e073ecd52c098', 'request_id': 'req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:55.115425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57abcbe1cc6049d0b04e073ecd52c098', 'request_id': 'req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:55.126469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] The token cache is being invalidate because project 57abcbe1cc6049d0b04e073ecd52c098 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:10:55.126760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:55.127385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-4e8875fc-d4bb-4553-9ab5-bc50335a3dbe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:55.128929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1372/2746] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:10:54 2022] DELETE /identity/v3/projects/57abcbe1cc6049d0b04e073ecd52c098 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:55.134044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd725ba4-3a36-464f-8722-3d6b2e22ba7c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.134515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd725ba4-3a36-464f-8722-3d6b2e22ba7c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.134614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd725ba4-3a36-464f-8722-3d6b2e22ba7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.153164 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-fd725ba4-3a36-464f-8722-3d6b2e22ba7c None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:55.154330 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-fd725ba4-3a36-464f-8722-3d6b2e22ba7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:55.211793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1375/2747] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:55.221034 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00ae8c-0c2c-49a1-98bd-dcf6fc5bc8cf None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.221324 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00ae8c-0c2c-49a1-98bd-dcf6fc5bc8cf None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.221643 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d00ae8c-0c2c-49a1-98bd-dcf6fc5bc8cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.240480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6d00ae8c-0c2c-49a1-98bd-dcf6fc5bc8cf None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:55.241711 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6d00ae8c-0c2c-49a1-98bd-dcf6fc5bc8cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:55.249458 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1373/2748] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:55.253590 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd725ba4-3a36-464f-8722-3d6b2e22ba7c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.263676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd725ba4-3a36-464f-8722-3d6b2e22ba7c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.268209 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045d3b1-1ee5-4967-9967-6f94053544c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.268854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045d3b1-1ee5-4967-9967-6f94053544c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.269134 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045d3b1-1ee5-4967-9967-6f94053544c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.269410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1045d3b1-1ee5-4967-9967-6f94053544c8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.270156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045d3b1-1ee5-4967-9967-6f94053544c8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:10:55.270463 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045d3b1-1ee5-4967-9967-6f94053544c8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.272036 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1045d3b1-1ee5-4967-9967-6f94053544c8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.283909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1376/2749] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:10:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:10:55.288132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00ae8c-0c2c-49a1-98bd-dcf6fc5bc8cf None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.297377 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d00ae8c-0c2c-49a1-98bd-dcf6fc5bc8cf None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.302009 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db6fca-5b30-4a4b-a056-f609cb8a3b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.302734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db6fca-5b30-4a4b-a056-f609cb8a3b95 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.302949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db6fca-5b30-4a4b-a056-f609cb8a3b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.303373 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9db6fca-5b30-4a4b-a056-f609cb8a3b95 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.304205 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db6fca-5b30-4a4b-a056-f609cb8a3b95 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.306618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9db6fca-5b30-4a4b-a056-f609cb8a3b95 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.334242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1374/2750] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:55 2022] POST /identity/v3/projects => generated 385 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:10:55.339173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045d3b1-1ee5-4967-9967-6f94053544c8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.349834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1045d3b1-1ee5-4967-9967-6f94053544c8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.405896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16637dee-59ec-48c5-9373-e2db06e10f2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.406490 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16637dee-59ec-48c5-9373-e2db06e10f2d admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.406776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16637dee-59ec-48c5-9373-e2db06e10f2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.407055 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16637dee-59ec-48c5-9373-e2db06e10f2d admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.407752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16637dee-59ec-48c5-9373-e2db06e10f2d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.409708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16637dee-59ec-48c5-9373-e2db06e10f2d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.429159 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1377/2751] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:55 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:55.433659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db6fca-5b30-4a4b-a056-f609cb8a3b95 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.443710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9db6fca-5b30-4a4b-a056-f609cb8a3b95 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.448932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3232c-d2d3-45da-927e-0f19be74a31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.449730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3232c-d2d3-45da-927e-0f19be74a31e admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.450032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3232c-d2d3-45da-927e-0f19be74a31e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.450305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd3232c-d2d3-45da-927e-0f19be74a31e admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.451020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3232c-d2d3-45da-927e-0f19be74a31e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.452609 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd3232c-d2d3-45da-927e-0f19be74a31e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.462710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1375/2752] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:55.467455 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16637dee-59ec-48c5-9373-e2db06e10f2d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.477423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16637dee-59ec-48c5-9373-e2db06e10f2d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.482143 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9c89d8-b4e4-4370-a3e7-1f5c687288c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.483324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c89d8-b4e4-4370-a3e7-1f5c687288c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.483324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c89d8-b4e4-4370-a3e7-1f5c687288c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.483601 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9c89d8-b4e4-4370-a3e7-1f5c687288c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.484323 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c89d8-b4e4-4370-a3e7-1f5c687288c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.485944 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9c89d8-b4e4-4370-a3e7-1f5c687288c4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.497013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1378/2753] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:10:55.501115 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3232c-d2d3-45da-927e-0f19be74a31e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.512059 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd3232c-d2d3-45da-927e-0f19be74a31e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.517030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6109f13-ea55-4715-b70b-add877906518 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.517945 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6109f13-ea55-4715-b70b-add877906518 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.518226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6109f13-ea55-4715-b70b-add877906518 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.518500 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6109f13-ea55-4715-b70b-add877906518 admin admin] PATH_INFO: `/v3/projects/12b7f53bb6094cbcab59ce6f6e860fef/users/b24591135f7a41059e03545f1ce0cfb2/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.533678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6109f13-ea55-4715-b70b-add877906518 admin admin] RBAC: Authorizing `identity:create_grant(project_id=12b7f53bb6094cbcab59ce6f6e860fef, user_id=b24591135f7a41059e03545f1ce0cfb2, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.536320 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6109f13-ea55-4715-b70b-add877906518 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.548783 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1376/2754] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:55 2022] PUT /identity/v3/projects/12b7f53bb6094cbcab59ce6f6e860fef/users/b24591135f7a41059e03545f1ce0cfb2/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:55.559027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd3896-3866-45ac-b072-831de9cda150 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.559339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd3896-3866-45ac-b072-831de9cda150 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.559620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cd3896-3866-45ac-b072-831de9cda150 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.587495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-b1cd3896-3866-45ac-b072-831de9cda150 None None] MFA Rules not processed for user `b24591135f7a41059e03545f1ce0cfb2`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:55.589384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b1cd3896-3866-45ac-b072-831de9cda150 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:55.630333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:55.655727 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1379/2755] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:55.660375 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6109f13-ea55-4715-b70b-add877906518 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.704154 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:55.723186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6109f13-ea55-4715-b70b-add877906518 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.727763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd28514-4e80-4c96-9e9d-6e6ceff29f28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.728437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd28514-4e80-4c96-9e9d-6e6ceff29f28 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.728715 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd28514-4e80-4c96-9e9d-6e6ceff29f28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.729029 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd28514-4e80-4c96-9e9d-6e6ceff29f28 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.729784 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd28514-4e80-4c96-9e9d-6e6ceff29f28 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.731683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd28514-4e80-4c96-9e9d-6e6ceff29f28 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.757450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1377/2756] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:55 2022] POST /identity/v3/projects => generated 383 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:55.762177 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd3896-3866-45ac-b072-831de9cda150 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.772604 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cd3896-3866-45ac-b072-831de9cda150 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.827674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b10d23-b213-4a7e-88b3-75922488914b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.828338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b10d23-b213-4a7e-88b3-75922488914b admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.828622 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b10d23-b213-4a7e-88b3-75922488914b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.828897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b10d23-b213-4a7e-88b3-75922488914b admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.829676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b10d23-b213-4a7e-88b3-75922488914b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.831572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93b10d23-b213-4a7e-88b3-75922488914b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.850603 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1380/2757] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:55 2022] POST /identity/v3/users => generated 292 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:55.854672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd28514-4e80-4c96-9e9d-6e6ceff29f28 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.865101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd28514-4e80-4c96-9e9d-6e6ceff29f28 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.870380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbe721-0b2f-4d92-857f-98f348407243 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.871043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe721-0b2f-4d92-857f-98f348407243 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.871327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe721-0b2f-4d92-857f-98f348407243 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.871604 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23cbe721-0b2f-4d92-857f-98f348407243 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.872326 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbe721-0b2f-4d92-857f-98f348407243 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.873985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23cbe721-0b2f-4d92-857f-98f348407243 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.884439 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1378/2758] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:55.888775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b10d23-b213-4a7e-88b3-75922488914b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.898964 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b10d23-b213-4a7e-88b3-75922488914b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.903689 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ba22bf-1078-4922-bef2-21d7a38a9ace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.904345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba22bf-1078-4922-bef2-21d7a38a9ace admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.904626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba22bf-1078-4922-bef2-21d7a38a9ace admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.904907 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ba22bf-1078-4922-bef2-21d7a38a9ace admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.905712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba22bf-1078-4922-bef2-21d7a38a9ace admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.907282 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ba22bf-1078-4922-bef2-21d7a38a9ace admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.917208 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1381/2759] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:55 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:55.921131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbe721-0b2f-4d92-857f-98f348407243 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:55.931171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23cbe721-0b2f-4d92-857f-98f348407243 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:55.936625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556b758-166b-47aa-8e87-95f0a90ced4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:55.937525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b758-166b-47aa-8e87-95f0a90ced4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.937836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b758-166b-47aa-8e87-95f0a90ced4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.938117 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5556b758-166b-47aa-8e87-95f0a90ced4e admin admin] PATH_INFO: `/v3/projects/55384dcb6fe94205b3cdc2a1b55c81d9/users/55fda6a2e5ff41fabbef847ab2df5693/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:55.953193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556b758-166b-47aa-8e87-95f0a90ced4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=55384dcb6fe94205b3cdc2a1b55c81d9, user_id=55fda6a2e5ff41fabbef847ab2df5693, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:55.955920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5556b758-166b-47aa-8e87-95f0a90ced4e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:55.968667 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1379/2760] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:55 2022] PUT /identity/v3/projects/55384dcb6fe94205b3cdc2a1b55c81d9/users/55fda6a2e5ff41fabbef847ab2df5693/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:55.979678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafc276-9b61-4bea-bfbc-b0232cc61c0a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:55.979969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafc276-9b61-4bea-bfbc-b0232cc61c0a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:55.980248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bafc276-9b61-4bea-bfbc-b0232cc61c0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:56.008146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-0bafc276-9b61-4bea-bfbc-b0232cc61c0a None None] MFA Rules not processed for user `55fda6a2e5ff41fabbef847ab2df5693`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:56.009369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0bafc276-9b61-4bea-bfbc-b0232cc61c0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:56.074147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1382/2761] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:10:56.078647 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556b758-166b-47aa-8e87-95f0a90ced4e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:56.139527 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5556b758-166b-47aa-8e87-95f0a90ced4e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:56.144073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4641b7a-8248-4cff-9c6a-91b9a04a8275 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:56.144719 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4641b7a-8248-4cff-9c6a-91b9a04a8275 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:56.145003 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4641b7a-8248-4cff-9c6a-91b9a04a8275 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:56.145275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4641b7a-8248-4cff-9c6a-91b9a04a8275 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:56.145988 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4641b7a-8248-4cff-9c6a-91b9a04a8275 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:56.147886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4641b7a-8248-4cff-9c6a-91b9a04a8275 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:56.172748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1380/2762] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:10:56 2022] POST /identity/v3/projects => generated 387 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:56.176764 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafc276-9b61-4bea-bfbc-b0232cc61c0a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:56.188208 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bafc276-9b61-4bea-bfbc-b0232cc61c0a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:56.234384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:56.245580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c4f5a-72c8-486f-aea1-03857882771f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:56.246244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c4f5a-72c8-486f-aea1-03857882771f admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:56.246537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c4f5a-72c8-486f-aea1-03857882771f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:56.246818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a1c4f5a-72c8-486f-aea1-03857882771f admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:56.247517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c4f5a-72c8-486f-aea1-03857882771f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:56.249438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a1c4f5a-72c8-486f-aea1-03857882771f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:56.268549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1383/2763] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:10:56 2022] POST /identity/v3/users => generated 300 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:10:56.272562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4641b7a-8248-4cff-9c6a-91b9a04a8275 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:56.282905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4641b7a-8248-4cff-9c6a-91b9a04a8275 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:56.287531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347be736-88e3-471e-a9d6-0c587697ea49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:56.288193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347be736-88e3-471e-a9d6-0c587697ea49 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:56.288481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347be736-88e3-471e-a9d6-0c587697ea49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:56.288764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-347be736-88e3-471e-a9d6-0c587697ea49 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:56.289489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347be736-88e3-471e-a9d6-0c587697ea49 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:56.291758 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-347be736-88e3-471e-a9d6-0c587697ea49 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:56.302451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1381/2764] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:56.307593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c4f5a-72c8-486f-aea1-03857882771f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:56.318703 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a1c4f5a-72c8-486f-aea1-03857882771f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:56.324020 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b666e-6f82-430b-bfc2-fe5cae72beca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:56.325072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b666e-6f82-430b-bfc2-fe5cae72beca admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:56.325353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b666e-6f82-430b-bfc2-fe5cae72beca admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:56.325736 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e4b666e-6f82-430b-bfc2-fe5cae72beca admin admin] PATH_INFO: `/v3/projects/3786fe0789c545beb5a12d26b9d3c7fc/users/24257ad5212242b8863928c748da8fe4/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:56.341424 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b666e-6f82-430b-bfc2-fe5cae72beca admin admin] RBAC: Authorizing `identity:create_grant(project_id=3786fe0789c545beb5a12d26b9d3c7fc, user_id=24257ad5212242b8863928c748da8fe4, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:56.344135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e4b666e-6f82-430b-bfc2-fe5cae72beca admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:56.358336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1384/2765] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:56 2022] PUT /identity/v3/projects/3786fe0789c545beb5a12d26b9d3c7fc/users/24257ad5212242b8863928c748da8fe4/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:10:56.362370 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347be736-88e3-471e-a9d6-0c587697ea49 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:56.373204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-347be736-88e3-471e-a9d6-0c587697ea49 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:56.429450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7369e824-8db1-4210-bf58-8a87cf8b981f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:56.430079 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369e824-8db1-4210-bf58-8a87cf8b981f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:56.430361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369e824-8db1-4210-bf58-8a87cf8b981f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:56.430637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7369e824-8db1-4210-bf58-8a87cf8b981f admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:56.431354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369e824-8db1-4210-bf58-8a87cf8b981f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:56.433083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7369e824-8db1-4210-bf58-8a87cf8b981f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:56.443041 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1382/2766] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:10:56 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:10:56.447367 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b666e-6f82-430b-bfc2-fe5cae72beca admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:56.457710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e4b666e-6f82-430b-bfc2-fe5cae72beca admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:56.462264 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81553f0c-e393-4ed4-90b3-a03326bb405f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:56.463745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553f0c-e393-4ed4-90b3-a03326bb405f admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:56.464169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553f0c-e393-4ed4-90b3-a03326bb405f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:56.464452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81553f0c-e393-4ed4-90b3-a03326bb405f admin admin] PATH_INFO: `/v3/projects/3786fe0789c545beb5a12d26b9d3c7fc/users/24257ad5212242b8863928c748da8fe4/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:56.466476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81553f0c-e393-4ed4-90b3-a03326bb405f admin admin] RBAC: Authorizing `identity:create_grant(project_id=3786fe0789c545beb5a12d26b9d3c7fc, user_id=24257ad5212242b8863928c748da8fe4, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:56.469094 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81553f0c-e393-4ed4-90b3-a03326bb405f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:56.479826 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1385/2767] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:10:56 2022] PUT /identity/v3/projects/3786fe0789c545beb5a12d26b9d3c7fc/users/24257ad5212242b8863928c748da8fe4/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:56.489978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba5a77-6229-41e3-82f1-ab0bb4ccb70d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:56.490270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba5a77-6229-41e3-82f1-ab0bb4ccb70d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:56.490553 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ba5a77-6229-41e3-82f1-ab0bb4ccb70d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:56.496607 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:56.519448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-89ba5a77-6229-41e3-82f1-ab0bb4ccb70d None None] MFA Rules not processed for user `24257ad5212242b8863928c748da8fe4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:56.520655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-89ba5a77-6229-41e3-82f1-ab0bb4ccb70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:56.596619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1383/2768] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:56 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:10:56.609128 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81553f0c-e393-4ed4-90b3-a03326bb405f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:56.609925 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-81553f0c-e393-4ed4-90b3-a03326bb405f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:56.668306 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81553f0c-e393-4ed4-90b3-a03326bb405f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:56.672684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9de65-ee89-4ab3-a7fb-872593252ced service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:56.673461 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9de65-ee89-4ab3-a7fb-872593252ced service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:56.673813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9de65-ee89-4ab3-a7fb-872593252ced service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:56.674155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebc9de65-ee89-4ab3-a7fb-872593252ced service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:56.684255 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9de65-ee89-4ab3-a7fb-872593252ced service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:56.685712 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebc9de65-ee89-4ab3-a7fb-872593252ced service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:56.698710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1386/2769] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:56 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:57.234720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba5a77-6229-41e3-82f1-ab0bb4ccb70d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:57.236113 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-89ba5a77-6229-41e3-82f1-ab0bb4ccb70d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:57.247930 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ba5a77-6229-41e3-82f1-ab0bb4ccb70d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:57.251486 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf98df3-a060-4585-aa83-5f70375ec49a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:57.252126 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf98df3-a060-4585-aa83-5f70375ec49a service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:57.252409 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf98df3-a060-4585-aa83-5f70375ec49a service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:57.252683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf98df3-a060-4585-aa83-5f70375ec49a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:57.310457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf98df3-a060-4585-aa83-5f70375ec49a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:57.311894 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf98df3-a060-4585-aa83-5f70375ec49a service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:57.321171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1384/2770] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:57 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:10:59.524314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9de65-ee89-4ab3-a7fb-872593252ced service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:59.526277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ebc9de65-ee89-4ab3-a7fb-872593252ced service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:59.590475 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebc9de65-ee89-4ab3-a7fb-872593252ced service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:59.594981 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:59.595841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:59.596125 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:59.596407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] PATH_INFO: `/v3/users/0a2973bc7bf0431980b9f3cb5a759479` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:59.597607 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0a2973bc7bf0431980b9f3cb5a759479)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:59.599515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:59.645339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0a2973bc7bf0431980b9f3cb5a759479'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:59.646498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:10:59.660728 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0a2973bc7bf0431980b9f3cb5a759479'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:59.668203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0a2973bc7bf0431980b9f3cb5a759479'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:59.668815 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0a2973bc7bf0431980b9f3cb5a759479'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:59.669410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0a2973bc7bf0431980b9f3cb5a759479'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:10:59.681911 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1387/2771] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:10:59 2022] DELETE /identity/v3/users/0a2973bc7bf0431980b9f3cb5a759479 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:10:59.686666 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413e205-3224-4763-81b8-1e4f96ebdc3d None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:59.687147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413e205-3224-4763-81b8-1e4f96ebdc3d None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:59.687237 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6413e205-3224-4763-81b8-1e4f96ebdc3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:59.704279 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6413e205-3224-4763-81b8-1e4f96ebdc3d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:10:59.705546 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6413e205-3224-4763-81b8-1e4f96ebdc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:59.764916 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1385/2772] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:10:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:10:59.774583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:10:59.776751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:10:59.847710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6020455-3521-44ee-91f1-a1abff6e6a05 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:10:59.851411 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2b33c-3001-4990-b2ff-f3b24410f649 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:10:59.852061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2b33c-3001-4990-b2ff-f3b24410f649 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:10:59.852342 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2b33c-3001-4990-b2ff-f3b24410f649 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:10:59.852695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e2b33c-3001-4990-b2ff-f3b24410f649 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:10:59.862468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2b33c-3001-4990-b2ff-f3b24410f649 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:10:59.863891 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e2b33c-3001-4990-b2ff-f3b24410f649 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:10:59.873669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1388/2773] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:10:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:00.434642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413e205-3224-4763-81b8-1e4f96ebdc3d None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:00.436121 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6413e205-3224-4763-81b8-1e4f96ebdc3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:00.449597 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6413e205-3224-4763-81b8-1e4f96ebdc3d None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:00.455147 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:00.456014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:00.456296 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:00.456566 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] PATH_INFO: `/v3/projects/dfdc42043200455faee589dcf1410a36` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:00.457780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] RBAC: Authorizing `identity:delete_project(project_id=dfdc42043200455faee589dcf1410a36)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:00.460499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:00.484297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:00.506981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dfdc42043200455faee589dcf1410a36', 'request_id': 'req-fe115f76-b93f-40d9-86be-b8b399ffa80a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:00.512710 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dfdc42043200455faee589dcf1410a36', 'request_id': 'req-fe115f76-b93f-40d9-86be-b8b399ffa80a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:00.519350 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dfdc42043200455faee589dcf1410a36', 'request_id': 'req-fe115f76-b93f-40d9-86be-b8b399ffa80a'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:00.530457 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] The token cache is being invalidate because project dfdc42043200455faee589dcf1410a36 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:00.530750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:00.531391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:00.532941 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1386/2774] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:00 2022] DELETE /identity/v3/projects/dfdc42043200455faee589dcf1410a36 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:00.536915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2b33c-3001-4990-b2ff-f3b24410f649 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:00.547238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e2b33c-3001-4990-b2ff-f3b24410f649 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:00.604870 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:00.605721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:00.606009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:00.606285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] PATH_INFO: `/v3/users/fea86cf17f2642fe8ca45cee98d9466e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:00.607516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fea86cf17f2642fe8ca45cee98d9466e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:00.609747 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:00.655966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fea86cf17f2642fe8ca45cee98d9466e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:00.663661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:00.669216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fea86cf17f2642fe8ca45cee98d9466e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:00.676893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fea86cf17f2642fe8ca45cee98d9466e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:00.677577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fea86cf17f2642fe8ca45cee98d9466e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:00.678178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-883165ba-05f9-4e4f-b943-8a09beca5bb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fea86cf17f2642fe8ca45cee98d9466e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:00.691338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1389/2775] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:00 2022] DELETE /identity/v3/users/fea86cf17f2642fe8ca45cee98d9466e => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:00.951308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:00.952747 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:00.966223 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe115f76-b93f-40d9-86be-b8b399ffa80a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.025592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.026605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.026897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.027206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] PATH_INFO: `/v3/projects/46450871949944509c4acc822c20a2cf` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.028395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46450871949944509c4acc822c20a2cf)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.030391 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.075885 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46450871949944509c4acc822c20a2cf', 'request_id': 'req-d5450979-e6ad-455e-87d3-302915b370e6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:01.081421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46450871949944509c4acc822c20a2cf', 'request_id': 'req-d5450979-e6ad-455e-87d3-302915b370e6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:01.088434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46450871949944509c4acc822c20a2cf', 'request_id': 'req-d5450979-e6ad-455e-87d3-302915b370e6'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:01.099275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] The token cache is being invalidate because project 46450871949944509c4acc822c20a2cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:01.099562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:01.100192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-d5450979-e6ad-455e-87d3-302915b370e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:01.101740 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1387/2776] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:00 2022] DELETE /identity/v3/projects/46450871949944509c4acc822c20a2cf => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:01.106766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676986f-279e-40ee-b0db-4320a640cbd2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.107054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676986f-279e-40ee-b0db-4320a640cbd2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.107327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7676986f-279e-40ee-b0db-4320a640cbd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.127015 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-7676986f-279e-40ee-b0db-4320a640cbd2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:01.128226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7676986f-279e-40ee-b0db-4320a640cbd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:01.190030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1390/2777] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:01.198235 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d89cb2-7e5f-409e-a4c2-d6bfd9c1c982 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.198527 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d89cb2-7e5f-409e-a4c2-d6bfd9c1c982 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.198807 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d89cb2-7e5f-409e-a4c2-d6bfd9c1c982 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.217135 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-32d89cb2-7e5f-409e-a4c2-d6bfd9c1c982 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:01.218434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-32d89cb2-7e5f-409e-a4c2-d6bfd9c1c982 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:01.226752 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1388/2778] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:01.231213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676986f-279e-40ee-b0db-4320a640cbd2 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.241090 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7676986f-279e-40ee-b0db-4320a640cbd2 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.246707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02c74e-4bff-4a93-bf1c-ca56a19b5b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.247364 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02c74e-4bff-4a93-bf1c-ca56a19b5b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.247659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02c74e-4bff-4a93-bf1c-ca56a19b5b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.247943 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c02c74e-4bff-4a93-bf1c-ca56a19b5b73 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.248671 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02c74e-4bff-4a93-bf1c-ca56a19b5b73 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:11:01.248986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02c74e-4bff-4a93-bf1c-ca56a19b5b73 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.250618 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c02c74e-4bff-4a93-bf1c-ca56a19b5b73 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.266064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1391/2779] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:11:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:11:01.267645 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d89cb2-7e5f-409e-a4c2-d6bfd9c1c982 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.277322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d89cb2-7e5f-409e-a4c2-d6bfd9c1c982 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.282336 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8c2c3-dc39-4ae1-b684-b8a141832922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.282994 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8c2c3-dc39-4ae1-b684-b8a141832922 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.283277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8c2c3-dc39-4ae1-b684-b8a141832922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.283550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd8c2c3-dc39-4ae1-b684-b8a141832922 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.284247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8c2c3-dc39-4ae1-b684-b8a141832922 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.286224 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd8c2c3-dc39-4ae1-b684-b8a141832922 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.317252 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1389/2780] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:01 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:01.323520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02c74e-4bff-4a93-bf1c-ca56a19b5b73 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.324244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8c2c3-dc39-4ae1-b684-b8a141832922 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.325733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-abd8c2c3-dc39-4ae1-b684-b8a141832922 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:01.333777 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c02c74e-4bff-4a93-bf1c-ca56a19b5b73 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.339473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=14,keystone:SELECT=83,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:01.392771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5776813-26b0-4aac-b970-83311cbf1d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.393097 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776813-26b0-4aac-b970-83311cbf1d73 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.393393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776813-26b0-4aac-b970-83311cbf1d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.393708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5776813-26b0-4aac-b970-83311cbf1d73 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.394504 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5776813-26b0-4aac-b970-83311cbf1d73 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.396383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5776813-26b0-4aac-b970-83311cbf1d73 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.397060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd8c2c3-dc39-4ae1-b684-b8a141832922 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.405748 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1bf1f37-ae2b-4df3-b5d9-63f43e76b2b5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.406751 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf1f37-ae2b-4df3-b5d9-63f43e76b2b5 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.407173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf1f37-ae2b-4df3-b5d9-63f43e76b2b5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.407587 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1bf1f37-ae2b-4df3-b5d9-63f43e76b2b5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.409491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d1bf1f37-ae2b-4df3-b5d9-63f43e76b2b5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:01.416910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1392/2781] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:01 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:01.421066 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5776813-26b0-4aac-b970-83311cbf1d73 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.430987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5776813-26b0-4aac-b970-83311cbf1d73 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.436522 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43247dd1-56b1-4de3-be90-ef62881fb2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.437170 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43247dd1-56b1-4de3-be90-ef62881fb2a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.437454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43247dd1-56b1-4de3-be90-ef62881fb2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.437793 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43247dd1-56b1-4de3-be90-ef62881fb2a1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.438517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43247dd1-56b1-4de3-be90-ef62881fb2a1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.440102 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43247dd1-56b1-4de3-be90-ef62881fb2a1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.450271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1393/2782] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:11:01.454147 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43247dd1-56b1-4de3-be90-ef62881fb2a1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.463882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43247dd1-56b1-4de3-be90-ef62881fb2a1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.468443 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76be76-13dd-4925-b8dc-998a867a2c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.469073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76be76-13dd-4925-b8dc-998a867a2c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.469356 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76be76-13dd-4925-b8dc-998a867a2c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.469668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76be76-13dd-4925-b8dc-998a867a2c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.470398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76be76-13dd-4925-b8dc-998a867a2c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.471952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d76be76-13dd-4925-b8dc-998a867a2c1f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.472180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf1f37-ae2b-4df3-b5d9-63f43e76b2b5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.473665 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1bf1f37-ae2b-4df3-b5d9-63f43e76b2b5 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.483292 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1394/2783] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:11:01.483730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1390/2784] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:01 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:01.487476 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76be76-13dd-4925-b8dc-998a867a2c1f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.498336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76be76-13dd-4925-b8dc-998a867a2c1f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.503072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5a312-7f7b-4415-8241-22c4f7505828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.503975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5a312-7f7b-4415-8241-22c4f7505828 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.504267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5a312-7f7b-4415-8241-22c4f7505828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.504548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa5a312-7f7b-4415-8241-22c4f7505828 admin admin] PATH_INFO: `/v3/projects/c657a439af1b46d0b985893770758a0a/users/43db48dd9aeb45fb8a3bfc021e3a6be4/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.519800 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5a312-7f7b-4415-8241-22c4f7505828 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c657a439af1b46d0b985893770758a0a, user_id=43db48dd9aeb45fb8a3bfc021e3a6be4, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.522758 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa5a312-7f7b-4415-8241-22c4f7505828 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.537242 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1395/2785] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:01 2022] PUT /identity/v3/projects/c657a439af1b46d0b985893770758a0a/users/43db48dd9aeb45fb8a3bfc021e3a6be4/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:01.599069 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1973b-877d-4fed-9910-d7246a88cf89 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.599387 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1973b-877d-4fed-9910-d7246a88cf89 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.599732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1973b-877d-4fed-9910-d7246a88cf89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.628638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-90c1973b-877d-4fed-9910-d7246a88cf89 None None] MFA Rules not processed for user `43db48dd9aeb45fb8a3bfc021e3a6be4`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:01.629879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-90c1973b-877d-4fed-9910-d7246a88cf89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:01.699684 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1391/2786] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:01.704260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5a312-7f7b-4415-8241-22c4f7505828 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.713730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=2,keystone:INSERT=5 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:01.775167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa5a312-7f7b-4415-8241-22c4f7505828 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.780233 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b2555-4fdd-4d40-b33f-63b88f3dddc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.780892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b2555-4fdd-4d40-b33f-63b88f3dddc6 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.781178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b2555-4fdd-4d40-b33f-63b88f3dddc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.781457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e69b2555-4fdd-4d40-b33f-63b88f3dddc6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.782248 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b2555-4fdd-4d40-b33f-63b88f3dddc6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.784156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e69b2555-4fdd-4d40-b33f-63b88f3dddc6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.810442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1396/2787] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:01 2022] POST /identity/v3/projects => generated 387 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:11:01.814508 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1973b-877d-4fed-9910-d7246a88cf89 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.826328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1973b-877d-4fed-9910-d7246a88cf89 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.880282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:01.888166 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260bcae-e3af-4c82-991b-7fcd5eabc5aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.888849 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260bcae-e3af-4c82-991b-7fcd5eabc5aa admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.889141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260bcae-e3af-4c82-991b-7fcd5eabc5aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.889422 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9260bcae-e3af-4c82-991b-7fcd5eabc5aa admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.890156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260bcae-e3af-4c82-991b-7fcd5eabc5aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.892093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9260bcae-e3af-4c82-991b-7fcd5eabc5aa admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.916054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1392/2788] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:01 2022] POST /identity/v3/users => generated 300 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:01.922057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b2555-4fdd-4d40-b33f-63b88f3dddc6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.932435 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e69b2555-4fdd-4d40-b33f-63b88f3dddc6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.937468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37be21-f7e2-47e1-8c31-43d131f7d625 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.938178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37be21-f7e2-47e1-8c31-43d131f7d625 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.938516 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37be21-f7e2-47e1-8c31-43d131f7d625 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.938706 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37be21-f7e2-47e1-8c31-43d131f7d625 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.939434 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37be21-f7e2-47e1-8c31-43d131f7d625 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.941078 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37be21-f7e2-47e1-8c31-43d131f7d625 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:01.952788 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1397/2789] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:01.956777 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260bcae-e3af-4c82-991b-7fcd5eabc5aa admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:01.967839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9260bcae-e3af-4c82-991b-7fcd5eabc5aa admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:01.972626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab347cf5-cbf3-4228-9695-19071c8504bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:01.973576 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab347cf5-cbf3-4228-9695-19071c8504bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:01.973891 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab347cf5-cbf3-4228-9695-19071c8504bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:01.974163 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab347cf5-cbf3-4228-9695-19071c8504bb admin admin] PATH_INFO: `/v3/projects/a02a56533dc543989599f3bee659ad4e/users/1a4a8219fcc448e98d10962e632041e5/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:01.990130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab347cf5-cbf3-4228-9695-19071c8504bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02a56533dc543989599f3bee659ad4e, user_id=1a4a8219fcc448e98d10962e632041e5, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:01.992778 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab347cf5-cbf3-4228-9695-19071c8504bb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:02.007453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1393/2790] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:01 2022] PUT /identity/v3/projects/a02a56533dc543989599f3bee659ad4e/users/1a4a8219fcc448e98d10962e632041e5/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:02.012244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37be21-f7e2-47e1-8c31-43d131f7d625 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:02.022984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37be21-f7e2-47e1-8c31-43d131f7d625 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:02.084666 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a9157-d5ed-4eb6-a8ed-30968d61386d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:02.085366 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a9157-d5ed-4eb6-a8ed-30968d61386d admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:02.085688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a9157-d5ed-4eb6-a8ed-30968d61386d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:02.085970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a9157-d5ed-4eb6-a8ed-30968d61386d admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:02.086799 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a9157-d5ed-4eb6-a8ed-30968d61386d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:02.089105 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a9157-d5ed-4eb6-a8ed-30968d61386d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:02.099245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1398/2791] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:02 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:02.103686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab347cf5-cbf3-4228-9695-19071c8504bb admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:02.115499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab347cf5-cbf3-4228-9695-19071c8504bb admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:02.120382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd91b7-aa87-4e52-a648-217ed13bbc09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:02.121352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd91b7-aa87-4e52-a648-217ed13bbc09 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:02.121820 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd91b7-aa87-4e52-a648-217ed13bbc09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:02.122112 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fd91b7-aa87-4e52-a648-217ed13bbc09 admin admin] PATH_INFO: `/v3/projects/a02a56533dc543989599f3bee659ad4e/users/1a4a8219fcc448e98d10962e632041e5/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:02.124230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd91b7-aa87-4e52-a648-217ed13bbc09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a02a56533dc543989599f3bee659ad4e, user_id=1a4a8219fcc448e98d10962e632041e5, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:02.127496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fd91b7-aa87-4e52-a648-217ed13bbc09 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:02.139146 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1394/2792] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:02 2022] PUT /identity/v3/projects/a02a56533dc543989599f3bee659ad4e/users/1a4a8219fcc448e98d10962e632041e5/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:02.150047 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ab0d9-595b-41fc-9025-f4846db492c4 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:02.150288 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ab0d9-595b-41fc-9025-f4846db492c4 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:02.150665 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366ab0d9-595b-41fc-9025-f4846db492c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:02.178123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-366ab0d9-595b-41fc-9025-f4846db492c4 None None] MFA Rules not processed for user `1a4a8219fcc448e98d10962e632041e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:02.179313 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-366ab0d9-595b-41fc-9025-f4846db492c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:02.248834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1399/2793] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:02 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:02.259618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd91b7-aa87-4e52-a648-217ed13bbc09 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:02.320143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fd91b7-aa87-4e52-a648-217ed13bbc09 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:02.325770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2ca865-0485-42b7-b590-441803adf02b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:02.326451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ca865-0485-42b7-b590-441803adf02b service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:02.326735 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ca865-0485-42b7-b590-441803adf02b service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:02.327011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2ca865-0485-42b7-b590-441803adf02b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:02.336674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ca865-0485-42b7-b590-441803adf02b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:02.339358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2ca865-0485-42b7-b590-441803adf02b service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:02.349202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1395/2794] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:02.475554 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ab0d9-595b-41fc-9025-f4846db492c4 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:02.480054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:02.540215 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366ab0d9-595b-41fc-9025-f4846db492c4 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:02.544857 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:02.545760 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:02.546048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:02.546328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] PATH_INFO: `/v3/users/43db48dd9aeb45fb8a3bfc021e3a6be4` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:02.547521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43db48dd9aeb45fb8a3bfc021e3a6be4)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:02.549457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:02.595073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43db48dd9aeb45fb8a3bfc021e3a6be4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:02.607062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43db48dd9aeb45fb8a3bfc021e3a6be4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:02.615473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43db48dd9aeb45fb8a3bfc021e3a6be4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:02.615701 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43db48dd9aeb45fb8a3bfc021e3a6be4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:02.616297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43db48dd9aeb45fb8a3bfc021e3a6be4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:02.629806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1400/2795] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:02 2022] DELETE /identity/v3/users/43db48dd9aeb45fb8a3bfc021e3a6be4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:02.634369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4cc27-a1d6-4709-8400-2aa6b8ee7990 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:02.634669 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4cc27-a1d6-4709-8400-2aa6b8ee7990 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:02.635759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea4cc27-a1d6-4709-8400-2aa6b8ee7990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:02.652979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-aea4cc27-a1d6-4709-8400-2aa6b8ee7990 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:02.654283 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aea4cc27-a1d6-4709-8400-2aa6b8ee7990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:02.711522 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1396/2796] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:02.720960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:02.722398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:02.779985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3d84bc-4bf8-4dc8-b430-8cf359891dd5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:02.783739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f1a5c-e28e-4d35-a3e2-fa569cbc2919 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:02.784384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f1a5c-e28e-4d35-a3e2-fa569cbc2919 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:02.784670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f1a5c-e28e-4d35-a3e2-fa569cbc2919 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:02.784950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a23f1a5c-e28e-4d35-a3e2-fa569cbc2919 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:02.794715 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f1a5c-e28e-4d35-a3e2-fa569cbc2919 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:02.796142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a23f1a5c-e28e-4d35-a3e2-fa569cbc2919 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:02.805610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1401/2797] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:03.069984 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4cc27-a1d6-4709-8400-2aa6b8ee7990 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:03.071489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-aea4cc27-a1d6-4709-8400-2aa6b8ee7990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:03.083790 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea4cc27-a1d6-4709-8400-2aa6b8ee7990 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:03.088463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:03.089352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:03.089667 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:03.089949 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] PATH_INFO: `/v3/projects/c657a439af1b46d0b985893770758a0a` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:03.091245 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] RBAC: Authorizing `identity:delete_project(project_id=c657a439af1b46d0b985893770758a0a)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:03.093438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:03.139548 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:03.142727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c657a439af1b46d0b985893770758a0a', 'request_id': 'req-24eeefab-00e0-4d48-bc65-f9154debcbdb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.149065 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c657a439af1b46d0b985893770758a0a', 'request_id': 'req-24eeefab-00e0-4d48-bc65-f9154debcbdb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.156932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c657a439af1b46d0b985893770758a0a', 'request_id': 'req-24eeefab-00e0-4d48-bc65-f9154debcbdb'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.169032 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] The token cache is being invalidate because project c657a439af1b46d0b985893770758a0a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:03.169323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.169987 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.171524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1397/2798] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:03 2022] DELETE /identity/v3/projects/c657a439af1b46d0b985893770758a0a => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:03.175851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f1a5c-e28e-4d35-a3e2-fa569cbc2919 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:03.185906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a23f1a5c-e28e-4d35-a3e2-fa569cbc2919 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:03.206205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:03.248023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:03.248898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:03.249201 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:03.249517 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] PATH_INFO: `/v3/users/1a4a8219fcc448e98d10962e632041e5` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:03.250708 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1a4a8219fcc448e98d10962e632041e5)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:03.252893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:03.301944 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1a4a8219fcc448e98d10962e632041e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.315301 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1a4a8219fcc448e98d10962e632041e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.324804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1a4a8219fcc448e98d10962e632041e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.325498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1a4a8219fcc448e98d10962e632041e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.326298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1a4a8219fcc448e98d10962e632041e5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.342398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1402/2799] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:03 2022] DELETE /identity/v3/users/1a4a8219fcc448e98d10962e632041e5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:03.603242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:03.605333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:03.619879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24eeefab-00e0-4d48-bc65-f9154debcbdb admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:03.630338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:03.631893 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:03.689562 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:03.690444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:03.690729 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:03.691014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] PATH_INFO: `/v3/users/b24591135f7a41059e03545f1ce0cfb2` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:03.692204 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b24591135f7a41059e03545f1ce0cfb2)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:03.694248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:03.710755 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c761a26f-037f-4dc5-bbe5-b2f2ea4e0daf admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:03.716260 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:03.717126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:03.717414 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:03.717726 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] PATH_INFO: `/v3/projects/a02a56533dc543989599f3bee659ad4e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:03.718996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a02a56533dc543989599f3bee659ad4e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:03.721238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:03.743643 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b24591135f7a41059e03545f1ce0cfb2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.756453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b24591135f7a41059e03545f1ce0cfb2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.764484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b24591135f7a41059e03545f1ce0cfb2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.765171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b24591135f7a41059e03545f1ce0cfb2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.765846 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-6f9dbaa1-1039-4ca8-b1e2-5e4dc5a172d7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b24591135f7a41059e03545f1ce0cfb2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.767745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:03.770829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a02a56533dc543989599f3bee659ad4e', 'request_id': 'req-ae285761-611c-4381-a663-d80d3d7a6fb2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.780307 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a02a56533dc543989599f3bee659ad4e', 'request_id': 'req-ae285761-611c-4381-a663-d80d3d7a6fb2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.781822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1398/2800] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:03 2022] DELETE /identity/v3/users/b24591135f7a41059e03545f1ce0cfb2 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:03.786495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a02a56533dc543989599f3bee659ad4e', 'request_id': 'req-ae285761-611c-4381-a663-d80d3d7a6fb2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.786690 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c3a75-bdda-4e58-8215-2b6784d0e17a None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:03.786982 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c3a75-bdda-4e58-8215-2b6784d0e17a None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:03.787261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283c3a75-bdda-4e58-8215-2b6784d0e17a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:03.799120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] The token cache is being invalidate because project a02a56533dc543989599f3bee659ad4e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:03.799412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.800035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae285761-611c-4381-a663-d80d3d7a6fb2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:03.801673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1403/2801] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:03 2022] DELETE /identity/v3/projects/a02a56533dc543989599f3bee659ad4e => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:03.805770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-283c3a75-bdda-4e58-8215-2b6784d0e17a None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:03.806293 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cc4ae-4b48-4c0c-afd7-eba20e0dc913 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:03.806588 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cc4ae-4b48-4c0c-afd7-eba20e0dc913 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:03.807003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709cc4ae-4b48-4c0c-afd7-eba20e0dc913 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:03.807136 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-283c3a75-bdda-4e58-8215-2b6784d0e17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:03.825684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-709cc4ae-4b48-4c0c-afd7-eba20e0dc913 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:03.826982 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-709cc4ae-4b48-4c0c-afd7-eba20e0dc913 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:03.856640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:03.872706 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1399/2802] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:03.882039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c3a75-bdda-4e58-8215-2b6784d0e17a None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:03.883450 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-283c3a75-bdda-4e58-8215-2b6784d0e17a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:03.891442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1404/2803] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:03.900079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a739c-270b-4590-95f2-3b7337c97774 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:03.900369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a739c-270b-4590-95f2-3b7337c97774 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:03.901205 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d26a739c-270b-4590-95f2-3b7337c97774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:03.920452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d26a739c-270b-4590-95f2-3b7337c97774 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:03.921686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d26a739c-270b-4590-95f2-3b7337c97774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:03.929840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1405/2804] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:03.933877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a739c-270b-4590-95f2-3b7337c97774 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:03.940496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283c3a75-bdda-4e58-8215-2b6784d0e17a None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:03.942851 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d26a739c-270b-4590-95f2-3b7337c97774 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:03.944220 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3198965-a4ff-4fa8-862f-54f659d07be1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:03.944959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3198965-a4ff-4fa8-862f-54f659d07be1 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:03.945214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3198965-a4ff-4fa8-862f-54f659d07be1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:03.945693 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3198965-a4ff-4fa8-862f-54f659d07be1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:03.947567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31363644-8dc0-4b8f-b9fd-a2156426fb97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:03.948371 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31363644-8dc0-4b8f-b9fd-a2156426fb97 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:03.948661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31363644-8dc0-4b8f-b9fd-a2156426fb97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:03.948933 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31363644-8dc0-4b8f-b9fd-a2156426fb97 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:03.949679 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31363644-8dc0-4b8f-b9fd-a2156426fb97 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:11:03.949984 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31363644-8dc0-4b8f-b9fd-a2156426fb97 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:03.951564 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31363644-8dc0-4b8f-b9fd-a2156426fb97 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:03.955525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3198965-a4ff-4fa8-862f-54f659d07be1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:03.956963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3198965-a4ff-4fa8-862f-54f659d07be1 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:03.963548 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1406/2805] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:11:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 20:11:03.967264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1400/2806] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:03.970429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31363644-8dc0-4b8f-b9fd-a2156426fb97 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:03.981284 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31363644-8dc0-4b8f-b9fd-a2156426fb97 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:03.986626 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830d432-6433-4366-9273-97f1a2cea8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:03.987283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830d432-6433-4366-9273-97f1a2cea8ba admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:03.987603 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830d432-6433-4366-9273-97f1a2cea8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:03.987882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b830d432-6433-4366-9273-97f1a2cea8ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:03.988577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830d432-6433-4366-9273-97f1a2cea8ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:03.990572 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b830d432-6433-4366-9273-97f1a2cea8ba admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.018246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1407/2807] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:03 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:11:04.022364 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3198965-a4ff-4fa8-862f-54f659d07be1 service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.034111 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3198965-a4ff-4fa8-862f-54f659d07be1 service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.094000 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341a974-d518-45a0-9df6-8e34b5ffe0db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.094776 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341a974-d518-45a0-9df6-8e34b5ffe0db admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.095158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341a974-d518-45a0-9df6-8e34b5ffe0db admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.095510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4341a974-d518-45a0-9df6-8e34b5ffe0db admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.096280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341a974-d518-45a0-9df6-8e34b5ffe0db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.098275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4341a974-d518-45a0-9df6-8e34b5ffe0db admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.124819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1401/2808] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:04 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:11:04.126444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830d432-6433-4366-9273-97f1a2cea8ba admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.137261 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b830d432-6433-4366-9273-97f1a2cea8ba admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.142139 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752443e-6cfe-40d9-920f-c4f28b73148f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.142980 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752443e-6cfe-40d9-920f-c4f28b73148f admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.143806 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752443e-6cfe-40d9-920f-c4f28b73148f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.144095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b752443e-6cfe-40d9-920f-c4f28b73148f admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.144832 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752443e-6cfe-40d9-920f-c4f28b73148f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.146481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b752443e-6cfe-40d9-920f-c4f28b73148f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.157542 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1408/2809] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:11:04.161470 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341a974-d518-45a0-9df6-8e34b5ffe0db admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.172459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4341a974-d518-45a0-9df6-8e34b5ffe0db admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.177066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2c67d-cb1d-4e01-975b-dc6406d149e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.177755 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c67d-cb1d-4e01-975b-dc6406d149e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.178043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c67d-cb1d-4e01-975b-dc6406d149e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.178328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc2c67d-cb1d-4e01-975b-dc6406d149e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.179045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2c67d-cb1d-4e01-975b-dc6406d149e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.180672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dc2c67d-cb1d-4e01-975b-dc6406d149e6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.190913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1402/2810] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:04.195033 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752443e-6cfe-40d9-920f-c4f28b73148f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.205302 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b752443e-6cfe-40d9-920f-c4f28b73148f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.210077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7472c09-b341-4a72-9d8d-827035af012e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.211110 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7472c09-b341-4a72-9d8d-827035af012e admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.211400 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7472c09-b341-4a72-9d8d-827035af012e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.211744 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7472c09-b341-4a72-9d8d-827035af012e admin admin] PATH_INFO: `/v3/projects/408afb5c5e3f48218163ce31f85e8a4f/users/1eadc8976c2143798b92be159b31b3db/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.232966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7472c09-b341-4a72-9d8d-827035af012e admin admin] RBAC: Authorizing `identity:create_grant(project_id=408afb5c5e3f48218163ce31f85e8a4f, user_id=1eadc8976c2143798b92be159b31b3db, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.236320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7472c09-b341-4a72-9d8d-827035af012e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.247106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2c67d-cb1d-4e01-975b-dc6406d149e6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.249021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-3dc2c67d-cb1d-4e01-975b-dc6406d149e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:04.259848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1409/2811] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:04 2022] PUT /identity/v3/projects/408afb5c5e3f48218163ce31f85e8a4f/users/1eadc8976c2143798b92be159b31b3db/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:04.262994 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f22a9-4dbb-450f-a808-a9ba2dcdc260 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.263297 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f22a9-4dbb-450f-a808-a9ba2dcdc260 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.263578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0f22a9-4dbb-450f-a808-a9ba2dcdc260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.279281 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc2c67d-cb1d-4e01-975b-dc6406d149e6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.290670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bf0f22a9-4dbb-450f-a808-a9ba2dcdc260 None None] MFA Rules not processed for user `1eadc8976c2143798b92be159b31b3db`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:04.291955 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bf0f22a9-4dbb-450f-a808-a9ba2dcdc260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:04.340406 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:04.342183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.343118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.343524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.343806 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] PATH_INFO: `/v3/projects/12b7f53bb6094cbcab59ce6f6e860fef` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.345012 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=12b7f53bb6094cbcab59ce6f6e860fef)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.347247 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.362615 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1410/2812] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:04.365293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:04.367326 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f22a9-4dbb-450f-a808-a9ba2dcdc260 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.378577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0f22a9-4dbb-450f-a808-a9ba2dcdc260 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.384002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81917d-21db-4137-be3d-2510ed06a3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.384648 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81917d-21db-4137-be3d-2510ed06a3df admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.385854 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81917d-21db-4137-be3d-2510ed06a3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.386035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da81917d-21db-4137-be3d-2510ed06a3df admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.387095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81917d-21db-4137-be3d-2510ed06a3df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.389120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da81917d-21db-4137-be3d-2510ed06a3df admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.398740 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '12b7f53bb6094cbcab59ce6f6e860fef', 'request_id': 'req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:04.406026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '12b7f53bb6094cbcab59ce6f6e860fef', 'request_id': 'req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:04.413733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '12b7f53bb6094cbcab59ce6f6e860fef', 'request_id': 'req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:04.418725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1411/2813] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:04 2022] POST /identity/v3/projects => generated 383 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:11:04.422655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81917d-21db-4137-be3d-2510ed06a3df admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.425772 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] The token cache is being invalidate because project 12b7f53bb6094cbcab59ce6f6e860fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:04.426051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:04.426872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:04.428703 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1403/2814] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:04 2022] DELETE /identity/v3/projects/12b7f53bb6094cbcab59ce6f6e860fef => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:04.432638 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.434051 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:04.485996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da81917d-21db-4137-be3d-2510ed06a3df admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.490602 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85585f9-e6fe-4148-96c4-eaf1227b3cb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.491276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85585f9-e6fe-4148-96c4-eaf1227b3cb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.491569 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85585f9-e6fe-4148-96c4-eaf1227b3cb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.491848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b85585f9-e6fe-4148-96c4-eaf1227b3cb9 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.492532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85585f9-e6fe-4148-96c4-eaf1227b3cb9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.495385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b85585f9-e6fe-4148-96c4-eaf1227b3cb9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.498613 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d92ebe2-aa09-4b8c-b4ba-6a0b255d72f2 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.503319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.504189 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.504472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.505066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] PATH_INFO: `/v3/users/55fda6a2e5ff41fabbef847ab2df5693` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.506261 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=55fda6a2e5ff41fabbef847ab2df5693)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.508046 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.517789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1412/2815] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:04 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:11:04.521734 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85585f9-e6fe-4148-96c4-eaf1227b3cb9 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.523977 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b85585f9-e6fe-4148-96c4-eaf1227b3cb9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:04.537381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b85585f9-e6fe-4148-96c4-eaf1227b3cb9 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.542612 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676ad55-f47d-40c0-b431-42ba6f7059d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.543303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676ad55-f47d-40c0-b431-42ba6f7059d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.543621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676ad55-f47d-40c0-b431-42ba6f7059d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.543938 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b676ad55-f47d-40c0-b431-42ba6f7059d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.544732 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676ad55-f47d-40c0-b431-42ba6f7059d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.546365 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b676ad55-f47d-40c0-b431-42ba6f7059d6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.556632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1413/2816] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:11:04.557047 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '55fda6a2e5ff41fabbef847ab2df5693'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:04.560759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676ad55-f47d-40c0-b431-42ba6f7059d6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.569228 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '55fda6a2e5ff41fabbef847ab2df5693'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:04.570212 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b676ad55-f47d-40c0-b431-42ba6f7059d6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.575710 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1d761-0961-434e-9ca1-ec8d74351758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.576569 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1d761-0961-434e-9ca1-ec8d74351758 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.576853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1d761-0961-434e-9ca1-ec8d74351758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.577128 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57d1d761-0961-434e-9ca1-ec8d74351758 admin admin] PATH_INFO: `/v3/projects/0b72d75597074564b5a2a403590fb4f9/users/7c0636afbd674f78a3d294738344230e/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.577236 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '55fda6a2e5ff41fabbef847ab2df5693'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:04.577954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '55fda6a2e5ff41fabbef847ab2df5693'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:04.578571 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '55fda6a2e5ff41fabbef847ab2df5693'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:04.593583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1d761-0961-434e-9ca1-ec8d74351758 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b72d75597074564b5a2a403590fb4f9, user_id=7c0636afbd674f78a3d294738344230e, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.593952 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1404/2817] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:04 2022] DELETE /identity/v3/users/55fda6a2e5ff41fabbef847ab2df5693 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:04.596238 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57d1d761-0961-434e-9ca1-ec8d74351758 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.609599 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1414/2818] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:04 2022] PUT /identity/v3/projects/0b72d75597074564b5a2a403590fb4f9/users/7c0636afbd674f78a3d294738344230e/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:04.613584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.615595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:04.654824 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:04.695788 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c8deeb-1317-4138-84ef-3a3d82199ac2 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.755920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b89eb022-ca26-4be4-888b-0ad44b544021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.756600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eb022-ca26-4be4-888b-0ad44b544021 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.756887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eb022-ca26-4be4-888b-0ad44b544021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.757167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b89eb022-ca26-4be4-888b-0ad44b544021 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.757928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89eb022-ca26-4be4-888b-0ad44b544021 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.759534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b89eb022-ca26-4be4-888b-0ad44b544021 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.770513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1405/2819] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:04 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:04.774868 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1d761-0961-434e-9ca1-ec8d74351758 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.788193 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57d1d761-0961-434e-9ca1-ec8d74351758 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.793122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f0735-3bc4-4c21-a6d8-0ff40b72221d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.794072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f0735-3bc4-4c21-a6d8-0ff40b72221d admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.794353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f0735-3bc4-4c21-a6d8-0ff40b72221d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.794655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da2f0735-3bc4-4c21-a6d8-0ff40b72221d admin admin] PATH_INFO: `/v3/projects/0b72d75597074564b5a2a403590fb4f9/users/7c0636afbd674f78a3d294738344230e/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.797157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f0735-3bc4-4c21-a6d8-0ff40b72221d admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b72d75597074564b5a2a403590fb4f9, user_id=7c0636afbd674f78a3d294738344230e, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.799916 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da2f0735-3bc4-4c21-a6d8-0ff40b72221d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:04.811578 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1415/2820] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:04 2022] PUT /identity/v3/projects/0b72d75597074564b5a2a403590fb4f9/users/7c0636afbd674f78a3d294738344230e/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:04.821867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fcbd7-60ef-46ce-bb18-31b1085b8f7f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.822165 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fcbd7-60ef-46ce-bb18-31b1085b8f7f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.822484 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15fcbd7-60ef-46ce-bb18-31b1085b8f7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.851175 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-a15fcbd7-60ef-46ce-bb18-31b1085b8f7f None None] MFA Rules not processed for user `7c0636afbd674f78a3d294738344230e`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:04.852388 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a15fcbd7-60ef-46ce-bb18-31b1085b8f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:04.898714 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f0735-3bc4-4c21-a6d8-0ff40b72221d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.901721 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-da2f0735-3bc4-4c21-a6d8-0ff40b72221d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:04.941362 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1406/2821] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:04 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:04.954138 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fcbd7-60ef-46ce-bb18-31b1085b8f7f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:04.955630 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-a15fcbd7-60ef-46ce-bb18-31b1085b8f7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:04.984511 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da2f0735-3bc4-4c21-a6d8-0ff40b72221d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:04.989629 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:04.990566 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:04.990997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:04.991266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] PATH_INFO: `/v3/projects/55384dcb6fe94205b3cdc2a1b55c81d9` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:04.992780 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=55384dcb6fe94205b3cdc2a1b55c81d9)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:04.995281 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:05.003352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:05.015315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15fcbd7-60ef-46ce-bb18-31b1085b8f7f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:05.018985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34990eb1-98eb-40f4-be23-d4f8b0386dbf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:05.019635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34990eb1-98eb-40f4-be23-d4f8b0386dbf service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.019950 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34990eb1-98eb-40f4-be23-d4f8b0386dbf service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.020226 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34990eb1-98eb-40f4-be23-d4f8b0386dbf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.027839 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:05.030598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34990eb1-98eb-40f4-be23-d4f8b0386dbf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:05.032447 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34990eb1-98eb-40f4-be23-d4f8b0386dbf service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:05.043157 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1407/2822] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:04 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:05.046245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '55384dcb6fe94205b3cdc2a1b55c81d9', 'request_id': 'req-20bf037c-3f54-4b86-bee9-5724052346b8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.052876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '55384dcb6fe94205b3cdc2a1b55c81d9', 'request_id': 'req-20bf037c-3f54-4b86-bee9-5724052346b8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.059428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '55384dcb6fe94205b3cdc2a1b55c81d9', 'request_id': 'req-20bf037c-3f54-4b86-bee9-5724052346b8'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.074178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] The token cache is being invalidate because project 55384dcb6fe94205b3cdc2a1b55c81d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:05.074887 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.075145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.076717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1416/2823] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:04 2022] DELETE /identity/v3/projects/55384dcb6fe94205b3cdc2a1b55c81d9 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:05.080628 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34990eb1-98eb-40f4-be23-d4f8b0386dbf service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:05.091974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34990eb1-98eb-40f4-be23-d4f8b0386dbf service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:05.113084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:05.114451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:05.151753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:05.152621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.152911 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.153192 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] PATH_INFO: `/v3/users/24257ad5212242b8863928c748da8fe4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.154444 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24257ad5212242b8863928c748da8fe4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:05.156371 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:05.172106 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20bf037c-3f54-4b86-bee9-5724052346b8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:05.176299 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8a5e-034b-4e11-8aa7-e535108d357b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:05.176985 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8a5e-034b-4e11-8aa7-e535108d357b service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.177269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8a5e-034b-4e11-8aa7-e535108d357b service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.177596 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769b8a5e-034b-4e11-8aa7-e535108d357b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.179445 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-769b8a5e-034b-4e11-8aa7-e535108d357b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:05.205101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24257ad5212242b8863928c748da8fe4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.218689 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24257ad5212242b8863928c748da8fe4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.226049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24257ad5212242b8863928c748da8fe4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.226760 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24257ad5212242b8863928c748da8fe4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.227394 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24257ad5212242b8863928c748da8fe4'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.241714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1408/2824] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:05 2022] DELETE /identity/v3/users/24257ad5212242b8863928c748da8fe4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:05.245454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b8a5e-034b-4e11-8aa7-e535108d357b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:05.247005 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769b8a5e-034b-4e11-8aa7-e535108d357b service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:05.256688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1417/2825] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:05 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:05.544407 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:05.545905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:05.553652 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8a5e-034b-4e11-8aa7-e535108d357b service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:05.555152 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-769b8a5e-034b-4e11-8aa7-e535108d357b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:05.570142 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769b8a5e-034b-4e11-8aa7-e535108d357b service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:05.620744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe1e372b-f5d8-41e8-9960-9970e021b8d3 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:05.623349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:05.625585 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:05.626767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.626767 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.626992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] PATH_INFO: `/v3/users/1eadc8976c2143798b92be159b31b3db` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.628437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] RBAC: Authorizing `identity:delete_user(user_id=1eadc8976c2143798b92be159b31b3db)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:05.630579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:05.635684 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:05.636791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.637173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.637632 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] PATH_INFO: `/v3/projects/3786fe0789c545beb5a12d26b9d3c7fc` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.639108 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3786fe0789c545beb5a12d26b9d3c7fc)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:05.641009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:05.646823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:05.679265 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1eadc8976c2143798b92be159b31b3db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.690683 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3786fe0789c545beb5a12d26b9d3c7fc', 'request_id': 'req-e8800517-5008-41b3-9d13-4228f8868fa5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.691670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1eadc8976c2143798b92be159b31b3db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.696876 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3786fe0789c545beb5a12d26b9d3c7fc', 'request_id': 'req-e8800517-5008-41b3-9d13-4228f8868fa5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.698763 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1eadc8976c2143798b92be159b31b3db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.699615 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1eadc8976c2143798b92be159b31b3db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.700229 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-152f5538-78fa-4fef-9c69-5457575b3bdf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1eadc8976c2143798b92be159b31b3db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.702532 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3786fe0789c545beb5a12d26b9d3c7fc', 'request_id': 'req-e8800517-5008-41b3-9d13-4228f8868fa5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.714589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1409/2826] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:05 2022] DELETE /identity/v3/users/1eadc8976c2143798b92be159b31b3db => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:05.717751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] The token cache is being invalidate because project 3786fe0789c545beb5a12d26b9d3c7fc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:05.717751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.717960 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e8800517-5008-41b3-9d13-4228f8868fa5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:05.719084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8841e65-cc65-46e1-994f-7288dedb33a9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.719378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8841e65-cc65-46e1-994f-7288dedb33a9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.719662 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8841e65-cc65-46e1-994f-7288dedb33a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.724277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1418/2827] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:05 2022] DELETE /identity/v3/projects/3786fe0789c545beb5a12d26b9d3c7fc => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:05.725568 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4f3c0-1d95-44d4-abce-7e3b8139efd2 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.725949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4f3c0-1d95-44d4-abce-7e3b8139efd2 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.726309 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba4f3c0-1d95-44d4-abce-7e3b8139efd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.747815 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-c8841e65-cc65-46e1-994f-7288dedb33a9 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:05.748978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c8841e65-cc65-46e1-994f-7288dedb33a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:05.776150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-3ba4f3c0-1d95-44d4-abce-7e3b8139efd2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:05.777351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-3ba4f3c0-1d95-44d4-abce-7e3b8139efd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:05.810348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1410/2828] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:05.820232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8841e65-cc65-46e1-994f-7288dedb33a9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:05.821267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c8841e65-cc65-46e1-994f-7288dedb33a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:05.841695 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1419/2829] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:05.850247 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db366521-00df-440e-aaa0-b319312916e8 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.850538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db366521-00df-440e-aaa0-b319312916e8 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.850813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db366521-00df-440e-aaa0-b319312916e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.870385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-db366521-00df-440e-aaa0-b319312916e8 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:05.871608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-db366521-00df-440e-aaa0-b319312916e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:05.879294 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8841e65-cc65-46e1-994f-7288dedb33a9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:05.879823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1420/2830] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:05.882958 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335fe2d-38f2-483f-b567-cd08c0b5d9bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:05.883605 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335fe2d-38f2-483f-b567-cd08c0b5d9bc service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.883724 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db366521-00df-440e-aaa0-b319312916e8 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:05.883886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335fe2d-38f2-483f-b567-cd08c0b5d9bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.884158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7335fe2d-38f2-483f-b567-cd08c0b5d9bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.893135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db366521-00df-440e-aaa0-b319312916e8 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:05.893459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335fe2d-38f2-483f-b567-cd08c0b5d9bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:05.894921 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7335fe2d-38f2-483f-b567-cd08c0b5d9bc service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:05.897796 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0c20e-77e8-4b7e-823f-23a33f083d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:05.898432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0c20e-77e8-4b7e-823f-23a33f083d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.898723 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0c20e-77e8-4b7e-823f-23a33f083d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.898997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad0c20e-77e8-4b7e-823f-23a33f083d2b admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.899715 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0c20e-77e8-4b7e-823f-23a33f083d2b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:11:05.900024 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0c20e-77e8-4b7e-823f-23a33f083d2b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:05.901877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad0c20e-77e8-4b7e-823f-23a33f083d2b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:05.904322 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1411/2831] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:05.913794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1421/2832] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:11:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 20:11:05.918653 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335fe2d-38f2-483f-b567-cd08c0b5d9bc service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:05.928312 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7335fe2d-38f2-483f-b567-cd08c0b5d9bc service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:05.934093 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb26c08-935e-4a68-92c4-2130dc99f852 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:05.934749 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb26c08-935e-4a68-92c4-2130dc99f852 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:05.935030 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb26c08-935e-4a68-92c4-2130dc99f852 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:05.935303 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afb26c08-935e-4a68-92c4-2130dc99f852 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:05.936016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb26c08-935e-4a68-92c4-2130dc99f852 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:05.937968 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afb26c08-935e-4a68-92c4-2130dc99f852 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:05.970681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:05.977808 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1412/2833] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:05 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:05.983008 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0c20e-77e8-4b7e-823f-23a33f083d2b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:05.994310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad0c20e-77e8-4b7e-823f-23a33f083d2b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.029410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:06.060565 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81317232-3664-4276-839c-7bf74db4337f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.061251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81317232-3664-4276-839c-7bf74db4337f admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.061560 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81317232-3664-4276-839c-7bf74db4337f admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.061844 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81317232-3664-4276-839c-7bf74db4337f admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.062558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81317232-3664-4276-839c-7bf74db4337f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.064505 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81317232-3664-4276-839c-7bf74db4337f admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.088381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1422/2834] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:05 2022] POST /identity/v3/users => generated 293 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:06.089600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb26c08-935e-4a68-92c4-2130dc99f852 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.100549 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afb26c08-935e-4a68-92c4-2130dc99f852 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.106332 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d98697-f8a0-4ff9-ac8b-228a2e6405cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.107021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d98697-f8a0-4ff9-ac8b-228a2e6405cc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.107307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d98697-f8a0-4ff9-ac8b-228a2e6405cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.107582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d98697-f8a0-4ff9-ac8b-228a2e6405cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.108582 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d98697-f8a0-4ff9-ac8b-228a2e6405cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.110248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d98697-f8a0-4ff9-ac8b-228a2e6405cc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.122488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1413/2835] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:06.126737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81317232-3664-4276-839c-7bf74db4337f admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.137835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81317232-3664-4276-839c-7bf74db4337f admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.142649 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c543c15-e865-424c-8d8c-aa128c496a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.143327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543c15-e865-424c-8d8c-aa128c496a54 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.143614 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543c15-e865-424c-8d8c-aa128c496a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.144073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c543c15-e865-424c-8d8c-aa128c496a54 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.144820 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c543c15-e865-424c-8d8c-aa128c496a54 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.146495 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c543c15-e865-424c-8d8c-aa128c496a54 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.161204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1423/2836] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:06.162405 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d98697-f8a0-4ff9-ac8b-228a2e6405cc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.173496 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d98697-f8a0-4ff9-ac8b-228a2e6405cc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.178441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbf74e-e421-4876-aecc-7f3090a37e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.179348 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbf74e-e421-4876-aecc-7f3090a37e93 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.179789 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbf74e-e421-4876-aecc-7f3090a37e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.180078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bdbf74e-e421-4876-aecc-7f3090a37e93 admin admin] PATH_INFO: `/v3/projects/9b356f1dc3da46f29ff520a738654490/users/b35950ffa29c46c0a3e9d02c896993b0/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.196438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbf74e-e421-4876-aecc-7f3090a37e93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b356f1dc3da46f29ff520a738654490, user_id=b35950ffa29c46c0a3e9d02c896993b0, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.199183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bdbf74e-e421-4876-aecc-7f3090a37e93 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.212649 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1414/2837] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:06 2022] PUT /identity/v3/projects/9b356f1dc3da46f29ff520a738654490/users/b35950ffa29c46c0a3e9d02c896993b0/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:06.225126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e91005-3f79-4b1f-960c-f0ceec0ac88c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.225610 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e91005-3f79-4b1f-960c-f0ceec0ac88c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.225927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e91005-3f79-4b1f-960c-f0ceec0ac88c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.255268 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-54e91005-3f79-4b1f-960c-f0ceec0ac88c None None] MFA Rules not processed for user `b35950ffa29c46c0a3e9d02c896993b0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:06.256515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-54e91005-3f79-4b1f-960c-f0ceec0ac88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:06.328611 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1424/2838] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:06.333650 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbf74e-e421-4876-aecc-7f3090a37e93 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.397680 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bdbf74e-e421-4876-aecc-7f3090a37e93 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.403073 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3002567-b924-4fe6-8dff-08cf6dd3e6ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.403842 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3002567-b924-4fe6-8dff-08cf6dd3e6ae admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.404193 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3002567-b924-4fe6-8dff-08cf6dd3e6ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.404541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3002567-b924-4fe6-8dff-08cf6dd3e6ae admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.405305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3002567-b924-4fe6-8dff-08cf6dd3e6ae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.407550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3002567-b924-4fe6-8dff-08cf6dd3e6ae admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.432426 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e91005-3f79-4b1f-960c-f0ceec0ac88c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.434124 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-54e91005-3f79-4b1f-960c-f0ceec0ac88c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:06.438246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1415/2839] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:06 2022] POST /identity/v3/projects => generated 385 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:06.439606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3002567-b924-4fe6-8dff-08cf6dd3e6ae admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.454975 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e91005-3f79-4b1f-960c-f0ceec0ac88c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.506191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3002567-b924-4fe6-8dff-08cf6dd3e6ae admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.512105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1aa2d-18a9-49e8-8931-d1ab625fe64e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.512847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1aa2d-18a9-49e8-8931-d1ab625fe64e admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.513196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1aa2d-18a9-49e8-8931-d1ab625fe64e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.513584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d1aa2d-18a9-49e8-8931-d1ab625fe64e admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.514340 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.515204 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.515487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.515682 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1aa2d-18a9-49e8-8931-d1ab625fe64e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.515797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] PATH_INFO: `/v3/projects/408afb5c5e3f48218163ce31f85e8a4f` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.516951 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=408afb5c5e3f48218163ce31f85e8a4f)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.517637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d1aa2d-18a9-49e8-8931-d1ab625fe64e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.518879 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.542520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1416/2840] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:06 2022] POST /identity/v3/users => generated 299 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:06.544052 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1aa2d-18a9-49e8-8931-d1ab625fe64e admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.555082 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d1aa2d-18a9-49e8-8931-d1ab625fe64e admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.565831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:06.567813 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '408afb5c5e3f48218163ce31f85e8a4f', 'request_id': 'req-a442a56d-87eb-4b82-83a4-588838b8a8b5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:06.578151 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '408afb5c5e3f48218163ce31f85e8a4f', 'request_id': 'req-a442a56d-87eb-4b82-83a4-588838b8a8b5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:06.584718 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '408afb5c5e3f48218163ce31f85e8a4f', 'request_id': 'req-a442a56d-87eb-4b82-83a4-588838b8a8b5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:06.596430 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:06.598899 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] The token cache is being invalidate because project 408afb5c5e3f48218163ce31f85e8a4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:06.599251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:06.600085 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:06.602029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1425/2841] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:06 2022] DELETE /identity/v3/projects/408afb5c5e3f48218163ce31f85e8a4f => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:06.605656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.607009 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:06.617720 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50081556-2714-4382-b121-dbd24945cb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.618469 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50081556-2714-4382-b121-dbd24945cb0d admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.618818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50081556-2714-4382-b121-dbd24945cb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.619162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50081556-2714-4382-b121-dbd24945cb0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.619953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50081556-2714-4382-b121-dbd24945cb0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.621569 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50081556-2714-4382-b121-dbd24945cb0d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.632374 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1417/2842] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:06.637232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50081556-2714-4382-b121-dbd24945cb0d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.639025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-50081556-2714-4382-b121-dbd24945cb0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:06.652156 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50081556-2714-4382-b121-dbd24945cb0d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.656779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa7afd-ed30-4cb5-9f78-c0b01b2f63e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.657813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7afd-ed30-4cb5-9f78-c0b01b2f63e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.658105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7afd-ed30-4cb5-9f78-c0b01b2f63e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.658382 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4aa7afd-ed30-4cb5-9f78-c0b01b2f63e2 admin admin] PATH_INFO: `/v3/projects/ff45bdd0ad2347bb982516f474ce78b3/users/875b8e6dc15b441a84f04ac2c9f48c82/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.673002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa7afd-ed30-4cb5-9f78-c0b01b2f63e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff45bdd0ad2347bb982516f474ce78b3, user_id=875b8e6dc15b441a84f04ac2c9f48c82, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.675692 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4aa7afd-ed30-4cb5-9f78-c0b01b2f63e2 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.676091 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a442a56d-87eb-4b82-83a4-588838b8a8b5 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.681058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.682144 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.682484 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.682858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] PATH_INFO: `/v3/users/7c0636afbd674f78a3d294738344230e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.684071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7c0636afbd674f78a3d294738344230e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.686266 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.690292 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1418/2843] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:06 2022] PUT /identity/v3/projects/ff45bdd0ad2347bb982516f474ce78b3/users/875b8e6dc15b441a84f04ac2c9f48c82/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:06.694241 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa7afd-ed30-4cb5-9f78-c0b01b2f63e2 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.739847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7c0636afbd674f78a3d294738344230e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:06.752175 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7c0636afbd674f78a3d294738344230e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:06.759280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4aa7afd-ed30-4cb5-9f78-c0b01b2f63e2 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.760501 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7c0636afbd674f78a3d294738344230e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:06.761289 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7c0636afbd674f78a3d294738344230e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:06.762004 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7c0636afbd674f78a3d294738344230e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:06.764521 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35ad3ae-4ece-4378-88ef-45030061a7c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.765301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ad3ae-4ece-4378-88ef-45030061a7c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.765618 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ad3ae-4ece-4378-88ef-45030061a7c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.765913 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35ad3ae-4ece-4378-88ef-45030061a7c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.766722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ad3ae-4ece-4378-88ef-45030061a7c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.768593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35ad3ae-4ece-4378-88ef-45030061a7c3 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.775382 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1426/2844] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:06 2022] DELETE /identity/v3/users/7c0636afbd674f78a3d294738344230e => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:06.779441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1419/2845] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:06 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:11:06.783446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:06.784866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:06.849462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:06.854487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d93d728-fb66-4b48-a34b-9e51f648e615 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:06.862745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9289855-1316-486a-ba23-7a23a6b3d74b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:06.864098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9289855-1316-486a-ba23-7a23a6b3d74b admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.864660 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9289855-1316-486a-ba23-7a23a6b3d74b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.865213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9289855-1316-486a-ba23-7a23a6b3d74b admin admin] PATH_INFO: `/v3/projects/ff45bdd0ad2347bb982516f474ce78b3/users/875b8e6dc15b441a84f04ac2c9f48c82/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.868081 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9289855-1316-486a-ba23-7a23a6b3d74b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ff45bdd0ad2347bb982516f474ce78b3, user_id=875b8e6dc15b441a84f04ac2c9f48c82, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:06.871817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9289855-1316-486a-ba23-7a23a6b3d74b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:06.883412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1427/2846] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:06 2022] PUT /identity/v3/projects/ff45bdd0ad2347bb982516f474ce78b3/users/875b8e6dc15b441a84f04ac2c9f48c82/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:06.893738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39ce80-c04d-4e2c-a2f9-0419f53280c5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:06.894164 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39ce80-c04d-4e2c-a2f9-0419f53280c5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:06.894513 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf39ce80-c04d-4e2c-a2f9-0419f53280c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:06.922966 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-cf39ce80-c04d-4e2c-a2f9-0419f53280c5 None None] MFA Rules not processed for user `875b8e6dc15b441a84f04ac2c9f48c82`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:06.924318 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cf39ce80-c04d-4e2c-a2f9-0419f53280c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:07.002317 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1420/2847] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:06 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:07.013276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9289855-1316-486a-ba23-7a23a6b3d74b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.015035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e9289855-1316-486a-ba23-7a23a6b3d74b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:07.066315 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39ce80-c04d-4e2c-a2f9-0419f53280c5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.068034 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cf39ce80-c04d-4e2c-a2f9-0419f53280c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:07.087189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9289855-1316-486a-ba23-7a23a6b3d74b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.093328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923fbc7-e599-46bd-ac87-421ccc77e3ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.094020 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923fbc7-e599-46bd-ac87-421ccc77e3ce service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.094310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923fbc7-e599-46bd-ac87-421ccc77e3ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.094593 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923fbc7-e599-46bd-ac87-421ccc77e3ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.105271 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923fbc7-e599-46bd-ac87-421ccc77e3ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:07.106875 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f923fbc7-e599-46bd-ac87-421ccc77e3ce service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:07.107730 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:07.118722 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1428/2848] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:07 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:07.143347 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf39ce80-c04d-4e2c-a2f9-0419f53280c5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.148343 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.149215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.149522 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.149819 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] PATH_INFO: `/v3/projects/0b72d75597074564b5a2a403590fb4f9` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.151631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b72d75597074564b5a2a403590fb4f9)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:07.153714 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:07.205200 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b72d75597074564b5a2a403590fb4f9', 'request_id': 'req-32c4c6c1-1f41-46a4-9460-47a35162f6f5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:07.210726 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b72d75597074564b5a2a403590fb4f9', 'request_id': 'req-32c4c6c1-1f41-46a4-9460-47a35162f6f5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:07.219389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b72d75597074564b5a2a403590fb4f9', 'request_id': 'req-32c4c6c1-1f41-46a4-9460-47a35162f6f5'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:07.231441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] The token cache is being invalidate because project 0b72d75597074564b5a2a403590fb4f9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:07.231853 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:07.232524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-32c4c6c1-1f41-46a4-9460-47a35162f6f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:07.234379 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1421/2849] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:07 2022] DELETE /identity/v3/projects/0b72d75597074564b5a2a403590fb4f9 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:07.239098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac214b1e-74c9-4f9f-94d9-244521a7e86e None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.239397 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac214b1e-74c9-4f9f-94d9-244521a7e86e None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.239681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac214b1e-74c9-4f9f-94d9-244521a7e86e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.258123 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-ac214b1e-74c9-4f9f-94d9-244521a7e86e None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:07.259339 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ac214b1e-74c9-4f9f-94d9-244521a7e86e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:07.341173 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1429/2850] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:07.354522 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb8753-56e9-41a5-8e4c-6a490625344f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.354818 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb8753-56e9-41a5-8e4c-6a490625344f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.355103 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdb8753-56e9-41a5-8e4c-6a490625344f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.373635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-1cdb8753-56e9-41a5-8e4c-6a490625344f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:07.378612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1cdb8753-56e9-41a5-8e4c-6a490625344f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:07.385568 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1422/2851] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:07.389831 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac214b1e-74c9-4f9f-94d9-244521a7e86e None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.406902 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac214b1e-74c9-4f9f-94d9-244521a7e86e None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.415543 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dc586-2a6a-4d65-8510-ec73f6b5e7f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.417069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dc586-2a6a-4d65-8510-ec73f6b5e7f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.417452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dc586-2a6a-4d65-8510-ec73f6b5e7f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.417862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-237dc586-2a6a-4d65-8510-ec73f6b5e7f1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.418836 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237dc586-2a6a-4d65-8510-ec73f6b5e7f1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:11:07.419239 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237dc586-2a6a-4d65-8510-ec73f6b5e7f1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:07.420918 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-237dc586-2a6a-4d65-8510-ec73f6b5e7f1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:07.435075 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1430/2852] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:11:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:11:07.439825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb8753-56e9-41a5-8e4c-6a490625344f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.450510 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdb8753-56e9-41a5-8e4c-6a490625344f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.456505 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24560714-ec94-47b2-86f0-2b78dd8ddf56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.457528 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24560714-ec94-47b2-86f0-2b78dd8ddf56 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.457959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24560714-ec94-47b2-86f0-2b78dd8ddf56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.458316 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24560714-ec94-47b2-86f0-2b78dd8ddf56 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.459344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24560714-ec94-47b2-86f0-2b78dd8ddf56 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:07.461679 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24560714-ec94-47b2-86f0-2b78dd8ddf56 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:07.497595 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1423/2853] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:07 2022] POST /identity/v3/projects => generated 375 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:07.498909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dc586-2a6a-4d65-8510-ec73f6b5e7f1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.510482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-237dc586-2a6a-4d65-8510-ec73f6b5e7f1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.569771 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa7154-1927-4c9e-ab9b-0c09da1cb894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.570477 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa7154-1927-4c9e-ab9b-0c09da1cb894 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.570765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa7154-1927-4c9e-ab9b-0c09da1cb894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.571044 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95aa7154-1927-4c9e-ab9b-0c09da1cb894 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.571733 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa7154-1927-4c9e-ab9b-0c09da1cb894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:07.574277 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95aa7154-1927-4c9e-ab9b-0c09da1cb894 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:07.594947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1431/2854] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:07 2022] POST /identity/v3/users => generated 288 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:07.599239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24560714-ec94-47b2-86f0-2b78dd8ddf56 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.610319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24560714-ec94-47b2-86f0-2b78dd8ddf56 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.615186 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f52a1-a75e-4b64-ac62-5b1474b29fbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.615883 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f52a1-a75e-4b64-ac62-5b1474b29fbb admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.616490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f52a1-a75e-4b64-ac62-5b1474b29fbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.616490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f70f52a1-a75e-4b64-ac62-5b1474b29fbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.617110 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f52a1-a75e-4b64-ac62-5b1474b29fbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:07.618774 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f70f52a1-a75e-4b64-ac62-5b1474b29fbb admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:07.636152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1424/2855] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:07.640328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa7154-1927-4c9e-ab9b-0c09da1cb894 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.650357 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95aa7154-1927-4c9e-ab9b-0c09da1cb894 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.655577 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89260f61-d4fd-4cbc-9ab0-fefaa5bfc93c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.656225 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89260f61-d4fd-4cbc-9ab0-fefaa5bfc93c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.656506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89260f61-d4fd-4cbc-9ab0-fefaa5bfc93c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.656805 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89260f61-d4fd-4cbc-9ab0-fefaa5bfc93c admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.657549 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89260f61-d4fd-4cbc-9ab0-fefaa5bfc93c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:07.659133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89260f61-d4fd-4cbc-9ab0-fefaa5bfc93c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:07.661444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:07.672976 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1432/2856] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:07 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:07.677702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f52a1-a75e-4b64-ac62-5b1474b29fbb admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.688852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f70f52a1-a75e-4b64-ac62-5b1474b29fbb admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.693635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860f0020-cc2e-4af3-91e0-1ce80b31ce08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.694596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f0020-cc2e-4af3-91e0-1ce80b31ce08 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.695001 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f0020-cc2e-4af3-91e0-1ce80b31ce08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.695337 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860f0020-cc2e-4af3-91e0-1ce80b31ce08 admin admin] PATH_INFO: `/v3/projects/8550e436c13f43d895ad4934a3ab2503/users/04d8e0ba1a284b98a667324dc3900c87/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.700441 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89260f61-d4fd-4cbc-9ab0-fefaa5bfc93c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.702150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-89260f61-d4fd-4cbc-9ab0-fefaa5bfc93c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:07.711429 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f0020-cc2e-4af3-91e0-1ce80b31ce08 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8550e436c13f43d895ad4934a3ab2503, user_id=04d8e0ba1a284b98a667324dc3900c87, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:07.714142 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860f0020-cc2e-4af3-91e0-1ce80b31ce08 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:07.714781 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89260f61-d4fd-4cbc-9ab0-fefaa5bfc93c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.719509 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.720375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.720658 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.720935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] PATH_INFO: `/v3/users/b35950ffa29c46c0a3e9d02c896993b0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.722153 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b35950ffa29c46c0a3e9d02c896993b0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:07.724063 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:07.728113 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1425/2857] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:07 2022] PUT /identity/v3/projects/8550e436c13f43d895ad4934a3ab2503/users/04d8e0ba1a284b98a667324dc3900c87/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:07.739579 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126a90d-0929-432a-ba3c-52eb9e1d2dae None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.739872 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126a90d-0929-432a-ba3c-52eb9e1d2dae None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.740152 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6126a90d-0929-432a-ba3c-52eb9e1d2dae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.768244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-6126a90d-0929-432a-ba3c-52eb9e1d2dae None None] MFA Rules not processed for user `04d8e0ba1a284b98a667324dc3900c87`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:07.769448 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6126a90d-0929-432a-ba3c-52eb9e1d2dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:07.781674 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b35950ffa29c46c0a3e9d02c896993b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:07.793673 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b35950ffa29c46c0a3e9d02c896993b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:07.800642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b35950ffa29c46c0a3e9d02c896993b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:07.801336 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b35950ffa29c46c0a3e9d02c896993b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:07.801992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-36b58887-52c9-45c2-83d5-ad7a6aa75a72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b35950ffa29c46c0a3e9d02c896993b0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:07.815827 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1433/2858] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:07 2022] DELETE /identity/v3/users/b35950ffa29c46c0a3e9d02c896993b0 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:07.820187 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34095945-6760-4aab-80de-79dc97c32802 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.820498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34095945-6760-4aab-80de-79dc97c32802 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.820776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34095945-6760-4aab-80de-79dc97c32802 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.820926 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:07.840398 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-34095945-6760-4aab-80de-79dc97c32802 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:07.841461 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1426/2859] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:07 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:07.841634 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-34095945-6760-4aab-80de-79dc97c32802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:07.853895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126a90d-0929-432a-ba3c-52eb9e1d2dae None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.855406 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-6126a90d-0929-432a-ba3c-52eb9e1d2dae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:07.901016 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1434/2860] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:07.911620 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34095945-6760-4aab-80de-79dc97c32802 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:07.912502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6126a90d-0929-432a-ba3c-52eb9e1d2dae None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.912987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-34095945-6760-4aab-80de-79dc97c32802 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:07.960897 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8581bac-2e10-4c34-8c0e-5772aceee641 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e46ec9c31094d03b6a413e09bc9f699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.961600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8581bac-2e10-4c34-8c0e-5772aceee641 service glance] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.961886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8581bac-2e10-4c34-8c0e-5772aceee641 service glance] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.962160 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8581bac-2e10-4c34-8c0e-5772aceee641 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.971345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34095945-6760-4aab-80de-79dc97c32802 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:07.975300 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f6d5e-276e-4212-aa47-d6b68bfb0a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:07.975947 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f6d5e-276e-4212-aa47-d6b68bfb0a1a service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:07.976237 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f6d5e-276e-4212-aa47-d6b68bfb0a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:07.976521 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12f6d5e-276e-4212-aa47-d6b68bfb0a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:07.986263 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f6d5e-276e-4212-aa47-d6b68bfb0a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:07.987707 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12f6d5e-276e-4212-aa47-d6b68bfb0a1a service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:07.998115 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1435/2861] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:08.022849 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8581bac-2e10-4c34-8c0e-5772aceee641 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:08.024305 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8581bac-2e10-4c34-8c0e-5772aceee641 service glance] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:08.033369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1427/2862] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:08.277395 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f6d5e-276e-4212-aa47-d6b68bfb0a1a service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:08.279285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-b12f6d5e-276e-4212-aa47-d6b68bfb0a1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:08.293216 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12f6d5e-276e-4212-aa47-d6b68bfb0a1a service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:08.299174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:08.299460 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:08.299752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:08.300046 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] PATH_INFO: `/v3/projects/9b356f1dc3da46f29ff520a738654490` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:08.301219 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b356f1dc3da46f29ff520a738654490)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:08.303236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:08.315002 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:08.365086 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b356f1dc3da46f29ff520a738654490', 'request_id': 'req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:08.372871 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b356f1dc3da46f29ff520a738654490', 'request_id': 'req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:08.378848 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b356f1dc3da46f29ff520a738654490', 'request_id': 'req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:08.391429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] The token cache is being invalidate because project 9b356f1dc3da46f29ff520a738654490 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:08.391717 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:08.392914 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:08.394196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1436/2863] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:08 2022] DELETE /identity/v3/projects/9b356f1dc3da46f29ff520a738654490 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:08.398213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8581bac-2e10-4c34-8c0e-5772aceee641 service glance] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:08.410705 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8581bac-2e10-4c34-8c0e-5772aceee641 service glance] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:08.494596 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:08.495768 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:08.496190 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:08.496480 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] PATH_INFO: `/v3/users/875b8e6dc15b441a84f04ac2c9f48c82` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:08.497737 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=875b8e6dc15b441a84f04ac2c9f48c82)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:08.499686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:08.501358 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=100 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:08.549119 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '875b8e6dc15b441a84f04ac2c9f48c82'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:08.561213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '875b8e6dc15b441a84f04ac2c9f48c82'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:08.568131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '875b8e6dc15b441a84f04ac2c9f48c82'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:08.568801 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '875b8e6dc15b441a84f04ac2c9f48c82'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:08.569421 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-1970184e-91be-4cf1-ad72-10e02e981c71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '875b8e6dc15b441a84f04ac2c9f48c82'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:08.581959 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1428/2864] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:08 2022] DELETE /identity/v3/users/875b8e6dc15b441a84f04ac2c9f48c82 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:08.870077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:08.871835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:08.885013 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb4f300e-8021-45d6-83dc-58bfb6beb4bf admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:08.950986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:08.951774 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:08.952057 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:08.952334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] PATH_INFO: `/v3/projects/ff45bdd0ad2347bb982516f474ce78b3` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:08.953609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ff45bdd0ad2347bb982516f474ce78b3)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:08.955538 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.004222 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ff45bdd0ad2347bb982516f474ce78b3', 'request_id': 'req-ce81d425-826e-4878-a541-4d9e9e3d05c4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:09.010515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ff45bdd0ad2347bb982516f474ce78b3', 'request_id': 'req-ce81d425-826e-4878-a541-4d9e9e3d05c4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:09.016098 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ff45bdd0ad2347bb982516f474ce78b3', 'request_id': 'req-ce81d425-826e-4878-a541-4d9e9e3d05c4'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:09.028570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] The token cache is being invalidate because project ff45bdd0ad2347bb982516f474ce78b3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80210) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:09.028856 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:09.029470 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ce81d425-826e-4878-a541-4d9e9e3d05c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:09.031070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1437/2865] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:08 2022] DELETE /identity/v3/projects/ff45bdd0ad2347bb982516f474ce78b3 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:09.036365 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60483617-6f30-4abd-ba26-9cd48f795785 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.036661 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60483617-6f30-4abd-ba26-9cd48f795785 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.036951 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60483617-6f30-4abd-ba26-9cd48f795785 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.056136 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-60483617-6f30-4abd-ba26-9cd48f795785 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:09.058036 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-60483617-6f30-4abd-ba26-9cd48f795785 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:09.118369 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1429/2866] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:09.127095 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1256a0-3b2a-4667-ae2a-4e15bb123f4f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.127392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1256a0-3b2a-4667-ae2a-4e15bb123f4f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.127678 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1256a0-3b2a-4667-ae2a-4e15bb123f4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.147200 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-1e1256a0-3b2a-4667-ae2a-4e15bb123f4f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:09.148423 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1e1256a0-3b2a-4667-ae2a-4e15bb123f4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:09.157353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1438/2867] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:09.161764 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60483617-6f30-4abd-ba26-9cd48f795785 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.172297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60483617-6f30-4abd-ba26-9cd48f795785 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.177033 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b00499-0ac7-4421-82bb-0cc94c259c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.177739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b00499-0ac7-4421-82bb-0cc94c259c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.178028 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b00499-0ac7-4421-82bb-0cc94c259c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.178308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b00499-0ac7-4421-82bb-0cc94c259c7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.179044 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b00499-0ac7-4421-82bb-0cc94c259c7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80211) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:11:09.179355 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b00499-0ac7-4421-82bb-0cc94c259c7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.180993 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76b00499-0ac7-4421-82bb-0cc94c259c7f admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.193057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1430/2868] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:11:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:11:09.197659 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1256a0-3b2a-4667-ae2a-4e15bb123f4f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.207120 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1256a0-3b2a-4667-ae2a-4e15bb123f4f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.211759 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82477ed-0afc-464d-801b-c02bbf6ebda2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.212403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82477ed-0afc-464d-801b-c02bbf6ebda2 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.212681 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82477ed-0afc-464d-801b-c02bbf6ebda2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.212954 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f82477ed-0afc-464d-801b-c02bbf6ebda2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.214191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82477ed-0afc-464d-801b-c02bbf6ebda2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.216073 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f82477ed-0afc-464d-801b-c02bbf6ebda2 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.237131 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:09.249810 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1439/2869] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:09 2022] POST /identity/v3/projects => generated 377 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:09.251216 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b00499-0ac7-4421-82bb-0cc94c259c7f admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.261535 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b00499-0ac7-4421-82bb-0cc94c259c7f admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.324574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d25f35-4fab-4c66-9f90-8ff6d750a86a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.325258 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d25f35-4fab-4c66-9f90-8ff6d750a86a admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.325598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d25f35-4fab-4c66-9f90-8ff6d750a86a admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.325879 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3d25f35-4fab-4c66-9f90-8ff6d750a86a admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.326598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d25f35-4fab-4c66-9f90-8ff6d750a86a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.328545 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3d25f35-4fab-4c66-9f90-8ff6d750a86a admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.338849 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:09.353702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1431/2870] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:09 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:09.356823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82477ed-0afc-464d-801b-c02bbf6ebda2 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.367324 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f82477ed-0afc-464d-801b-c02bbf6ebda2 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.372023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918b6bd-e19c-4d9b-840f-f080f5024803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.372735 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918b6bd-e19c-4d9b-840f-f080f5024803 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.373042 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918b6bd-e19c-4d9b-840f-f080f5024803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.373345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b918b6bd-e19c-4d9b-840f-f080f5024803 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.374163 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918b6bd-e19c-4d9b-840f-f080f5024803 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.375900 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b918b6bd-e19c-4d9b-840f-f080f5024803 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.387682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1440/2871] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:09.391780 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d25f35-4fab-4c66-9f90-8ff6d750a86a admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.402555 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3d25f35-4fab-4c66-9f90-8ff6d750a86a admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.407481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-103920f4-f5f1-4d03-be9d-093d428490ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.408145 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103920f4-f5f1-4d03-be9d-093d428490ce admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.408434 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103920f4-f5f1-4d03-be9d-093d428490ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.408709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-103920f4-f5f1-4d03-be9d-093d428490ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.409437 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103920f4-f5f1-4d03-be9d-093d428490ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.411130 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-103920f4-f5f1-4d03-be9d-093d428490ce admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.423195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1432/2872] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:09.428043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918b6bd-e19c-4d9b-840f-f080f5024803 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.439274 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b918b6bd-e19c-4d9b-840f-f080f5024803 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.444720 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6243a8-a93f-4b34-9cba-fb2340266a1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.445633 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6243a8-a93f-4b34-9cba-fb2340266a1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.445920 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6243a8-a93f-4b34-9cba-fb2340266a1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.446203 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6243a8-a93f-4b34-9cba-fb2340266a1c admin admin] PATH_INFO: `/v3/projects/27f326d58b3d448885e835629d352dc1/users/e19f72890e9f4e98962e3c0f30a814e0/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.461258 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6243a8-a93f-4b34-9cba-fb2340266a1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f326d58b3d448885e835629d352dc1, user_id=e19f72890e9f4e98962e3c0f30a814e0, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.464506 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6243a8-a93f-4b34-9cba-fb2340266a1c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.477606 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1441/2873] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:09 2022] PUT /identity/v3/projects/27f326d58b3d448885e835629d352dc1/users/e19f72890e9f4e98962e3c0f30a814e0/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:09.489953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f4a53-01ff-4cc9-b11a-30ba3879f08f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.490242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f4a53-01ff-4cc9-b11a-30ba3879f08f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.490533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898f4a53-01ff-4cc9-b11a-30ba3879f08f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.518887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-898f4a53-01ff-4cc9-b11a-30ba3879f08f None None] MFA Rules not processed for user `e19f72890e9f4e98962e3c0f30a814e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:09.520088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-898f4a53-01ff-4cc9-b11a-30ba3879f08f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:09.589349 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1433/2874] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:09.593711 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6243a8-a93f-4b34-9cba-fb2340266a1c admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.655835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6243a8-a93f-4b34-9cba-fb2340266a1c admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.660561 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75893c-f9f1-4f7c-8c63-df092fb537f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.661319 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75893c-f9f1-4f7c-8c63-df092fb537f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.661647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75893c-f9f1-4f7c-8c63-df092fb537f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.661924 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da75893c-f9f1-4f7c-8c63-df092fb537f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.662662 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75893c-f9f1-4f7c-8c63-df092fb537f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.664692 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da75893c-f9f1-4f7c-8c63-df092fb537f4 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.691944 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1442/2875] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:09 2022] POST /identity/v3/projects => generated 377 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:11:09.695627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f4a53-01ff-4cc9-b11a-30ba3879f08f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.706493 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898f4a53-01ff-4cc9-b11a-30ba3879f08f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.763014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457845c-1b8e-4d3c-9d6f-710c3af90341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.763601 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457845c-1b8e-4d3c-9d6f-710c3af90341 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.763886 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457845c-1b8e-4d3c-9d6f-710c3af90341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.764161 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457845c-1b8e-4d3c-9d6f-710c3af90341 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.764833 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457845c-1b8e-4d3c-9d6f-710c3af90341 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.766792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5457845c-1b8e-4d3c-9d6f-710c3af90341 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.788184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1434/2876] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:09 2022] POST /identity/v3/users => generated 289 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:09.792853 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75893c-f9f1-4f7c-8c63-df092fb537f4 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.803062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da75893c-f9f1-4f7c-8c63-df092fb537f4 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.808410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0119b38-8993-45fc-bced-eee39c599e5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.809083 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119b38-8993-45fc-bced-eee39c599e5a admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.809384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119b38-8993-45fc-bced-eee39c599e5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.809687 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0119b38-8993-45fc-bced-eee39c599e5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.810425 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0119b38-8993-45fc-bced-eee39c599e5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.811992 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0119b38-8993-45fc-bced-eee39c599e5a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.822317 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1443/2877] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:09.826067 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457845c-1b8e-4d3c-9d6f-710c3af90341 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.836176 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457845c-1b8e-4d3c-9d6f-710c3af90341 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.841378 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24aa0e1-0a12-40c2-ad1a-62ddca3fe141 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.842123 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24aa0e1-0a12-40c2-ad1a-62ddca3fe141 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.842408 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24aa0e1-0a12-40c2-ad1a-62ddca3fe141 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.842716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24aa0e1-0a12-40c2-ad1a-62ddca3fe141 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.843454 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24aa0e1-0a12-40c2-ad1a-62ddca3fe141 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.845109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24aa0e1-0a12-40c2-ad1a-62ddca3fe141 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.856499 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1435/2878] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:09.860898 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0119b38-8993-45fc-bced-eee39c599e5a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:09.871700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0119b38-8993-45fc-bced-eee39c599e5a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:09.876368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc987f-29fa-4816-beb0-a7ffe9161813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:09.877244 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc987f-29fa-4816-beb0-a7ffe9161813 admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.877553 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc987f-29fa-4816-beb0-a7ffe9161813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.877862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8efc987f-29fa-4816-beb0-a7ffe9161813 admin admin] PATH_INFO: `/v3/projects/0180664745684a04ae7bab943d601f18/users/4a09e275eb5d455481034b534e6cc2a6/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.892840 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc987f-29fa-4816-beb0-a7ffe9161813 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0180664745684a04ae7bab943d601f18, user_id=4a09e275eb5d455481034b534e6cc2a6, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:09.895550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8efc987f-29fa-4816-beb0-a7ffe9161813 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:09.910165 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1444/2879] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:09 2022] PUT /identity/v3/projects/0180664745684a04ae7bab943d601f18/users/4a09e275eb5d455481034b534e6cc2a6/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:09.921045 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51fbff-72cb-4529-bc6b-7cbd6210f0b2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:09.921969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51fbff-72cb-4529-bc6b-7cbd6210f0b2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:09.922262 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f51fbff-72cb-4529-bc6b-7cbd6210f0b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:09.949173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-0f51fbff-72cb-4529-bc6b-7cbd6210f0b2 None None] MFA Rules not processed for user `4a09e275eb5d455481034b534e6cc2a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:09.950438 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0f51fbff-72cb-4529-bc6b-7cbd6210f0b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:09.997173 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:10.020432 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1436/2880] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:10.024909 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc987f-29fa-4816-beb0-a7ffe9161813 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:10.086567 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8efc987f-29fa-4816-beb0-a7ffe9161813 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:10.092250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd544c3-ad2e-448d-ac20-443d9b6d1692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:10.093079 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd544c3-ad2e-448d-ac20-443d9b6d1692 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:10.093366 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd544c3-ad2e-448d-ac20-443d9b6d1692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:10.093751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd544c3-ad2e-448d-ac20-443d9b6d1692 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:10.094464 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd544c3-ad2e-448d-ac20-443d9b6d1692 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:10.096823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd544c3-ad2e-448d-ac20-443d9b6d1692 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:10.117265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:10.128028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1445/2881] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:10 2022] POST /identity/v3/projects => generated 375 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:10.129169 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51fbff-72cb-4529-bc6b-7cbd6210f0b2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:10.140356 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f51fbff-72cb-4529-bc6b-7cbd6210f0b2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:10.198962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31ae70-6b4a-48db-906e-b0e927186e93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:10.199637 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31ae70-6b4a-48db-906e-b0e927186e93 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:10.199924 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31ae70-6b4a-48db-906e-b0e927186e93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:10.200211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb31ae70-6b4a-48db-906e-b0e927186e93 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:10.200896 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31ae70-6b4a-48db-906e-b0e927186e93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:10.202868 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb31ae70-6b4a-48db-906e-b0e927186e93 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:10.224078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1437/2882] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:10 2022] POST /identity/v3/users => generated 294 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:10.227823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd544c3-ad2e-448d-ac20-443d9b6d1692 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:10.239323 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd544c3-ad2e-448d-ac20-443d9b6d1692 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:10.243989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebb526-8929-45e1-ac2f-c027f90de7d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:10.244677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebb526-8929-45e1-ac2f-c027f90de7d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:10.244927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebb526-8929-45e1-ac2f-c027f90de7d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:10.245199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bebb526-8929-45e1-ac2f-c027f90de7d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:10.245946 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebb526-8929-45e1-ac2f-c027f90de7d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:10.247563 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bebb526-8929-45e1-ac2f-c027f90de7d1 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:10.257987 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1446/2883] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:10.262770 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31ae70-6b4a-48db-906e-b0e927186e93 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:10.272937 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb31ae70-6b4a-48db-906e-b0e927186e93 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:10.278344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c80e47-fa1b-45e1-8240-c63f550acdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:10.279231 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c80e47-fa1b-45e1-8240-c63f550acdf5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:10.279526 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c80e47-fa1b-45e1-8240-c63f550acdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:10.279799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c80e47-fa1b-45e1-8240-c63f550acdf5 admin admin] PATH_INFO: `/v3/projects/582eec9e1a1d4f169ba52a2426609ba4/users/11e1020dc82f4b7d91da386fc03eff92/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:10.295363 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c80e47-fa1b-45e1-8240-c63f550acdf5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582eec9e1a1d4f169ba52a2426609ba4, user_id=11e1020dc82f4b7d91da386fc03eff92, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:10.298083 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c80e47-fa1b-45e1-8240-c63f550acdf5 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:10.312180 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1438/2884] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:10 2022] PUT /identity/v3/projects/582eec9e1a1d4f169ba52a2426609ba4/users/11e1020dc82f4b7d91da386fc03eff92/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:10.316346 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebb526-8929-45e1-ac2f-c027f90de7d1 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:10.327315 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bebb526-8929-45e1-ac2f-c027f90de7d1 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:10.384782 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c8ee81-57ee-4345-90e6-787c27823826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:10.386696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8ee81-57ee-4345-90e6-787c27823826 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:10.386993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8ee81-57ee-4345-90e6-787c27823826 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:10.387280 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c8ee81-57ee-4345-90e6-787c27823826 admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:10.388014 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8ee81-57ee-4345-90e6-787c27823826 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:10.389664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c8ee81-57ee-4345-90e6-787c27823826 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:10.399685 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1447/2885] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:10 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:10.404738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c80e47-fa1b-45e1-8240-c63f550acdf5 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:10.415385 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c80e47-fa1b-45e1-8240-c63f550acdf5 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:10.420124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5b2d-ae8d-44ef-82f9-b9cdf5df7055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:10.421004 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5b2d-ae8d-44ef-82f9-b9cdf5df7055 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:10.421289 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5b2d-ae8d-44ef-82f9-b9cdf5df7055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:10.421672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0a5b2d-ae8d-44ef-82f9-b9cdf5df7055 admin admin] PATH_INFO: `/v3/projects/582eec9e1a1d4f169ba52a2426609ba4/users/11e1020dc82f4b7d91da386fc03eff92/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:10.423657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a5b2d-ae8d-44ef-82f9-b9cdf5df7055 admin admin] RBAC: Authorizing `identity:create_grant(project_id=582eec9e1a1d4f169ba52a2426609ba4, user_id=11e1020dc82f4b7d91da386fc03eff92, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:10.426368 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0a5b2d-ae8d-44ef-82f9-b9cdf5df7055 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:10.436616 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1439/2886] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:10 2022] PUT /identity/v3/projects/582eec9e1a1d4f169ba52a2426609ba4/users/11e1020dc82f4b7d91da386fc03eff92/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:10.448438 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c59bcc-f5b5-4cd7-84e7-89ee490d8a5c None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:10.448737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c59bcc-f5b5-4cd7-84e7-89ee490d8a5c None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:10.449019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c59bcc-f5b5-4cd7-84e7-89ee490d8a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:10.476383 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f5c59bcc-f5b5-4cd7-84e7-89ee490d8a5c None None] MFA Rules not processed for user `11e1020dc82f4b7d91da386fc03eff92`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:10.477642 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f5c59bcc-f5b5-4cd7-84e7-89ee490d8a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:10.546573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1448/2887] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:10 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:10.558531 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5b2d-ae8d-44ef-82f9-b9cdf5df7055 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:10.560443 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-ce0a5b2d-ae8d-44ef-82f9-b9cdf5df7055 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:10.623253 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0a5b2d-ae8d-44ef-82f9-b9cdf5df7055 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:10.626887 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257ee4c-34ec-4665-a98d-0dae383ea5d0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:10.627541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ee4c-34ec-4665-a98d-0dae383ea5d0 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:10.627825 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ee4c-34ec-4665-a98d-0dae383ea5d0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:10.628101 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d257ee4c-34ec-4665-a98d-0dae383ea5d0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:10.677288 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:10.688739 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257ee4c-34ec-4665-a98d-0dae383ea5d0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:10.690274 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d257ee4c-34ec-4665-a98d-0dae383ea5d0 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:10.699681 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1440/2888] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:10 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:12.185535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c59bcc-f5b5-4cd7-84e7-89ee490d8a5c None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:12.195457 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c59bcc-f5b5-4cd7-84e7-89ee490d8a5c None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:12.200053 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf934aa4-17ae-427f-ac55-0348600c9b8c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:12.200705 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf934aa4-17ae-427f-ac55-0348600c9b8c service cinder] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:12.201058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf934aa4-17ae-427f-ac55-0348600c9b8c service cinder] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:12.201407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf934aa4-17ae-427f-ac55-0348600c9b8c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:12.260948 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf934aa4-17ae-427f-ac55-0348600c9b8c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:12.262927 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf934aa4-17ae-427f-ac55-0348600c9b8c service cinder] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:12.273176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1449/2889] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:12 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:12.797060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257ee4c-34ec-4665-a98d-0dae383ea5d0 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:12.798547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d257ee4c-34ec-4665-a98d-0dae383ea5d0 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:12.810626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d257ee4c-34ec-4665-a98d-0dae383ea5d0 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:12.814488 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae414a70-7e11-4e9e-80b7-768dc89598a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:12.815141 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae414a70-7e11-4e9e-80b7-768dc89598a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:12.815436 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae414a70-7e11-4e9e-80b7-768dc89598a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:12.815734 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae414a70-7e11-4e9e-80b7-768dc89598a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:12.826476 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae414a70-7e11-4e9e-80b7-768dc89598a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:12.827920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae414a70-7e11-4e9e-80b7-768dc89598a1 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:12.837598 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1441/2890] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:12 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:16.686462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf934aa4-17ae-427f-ac55-0348600c9b8c service cinder] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:16.695082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:16.753910 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf934aa4-17ae-427f-ac55-0348600c9b8c service cinder] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:16.758482 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:16.759348 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:16.759686 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:16.759969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] PATH_INFO: `/v3/users/e19f72890e9f4e98962e3c0f30a814e0` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:16.761195 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e19f72890e9f4e98962e3c0f30a814e0)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:16.763196 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:16.811037 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e19f72890e9f4e98962e3c0f30a814e0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:16.822888 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e19f72890e9f4e98962e3c0f30a814e0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:16.829931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e19f72890e9f4e98962e3c0f30a814e0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:16.830605 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e19f72890e9f4e98962e3c0f30a814e0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:16.831210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e19f72890e9f4e98962e3c0f30a814e0'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:16.844730 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1450/2891] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:16 2022] DELETE /identity/v3/users/e19f72890e9f4e98962e3c0f30a814e0 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:16.849431 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4436d-3696-403f-94ab-7a1f5a961f13 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:16.849940 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4436d-3696-403f-94ab-7a1f5a961f13 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:16.850060 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba4436d-3696-403f-94ab-7a1f5a961f13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:16.867836 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4ba4436d-3696-403f-94ab-7a1f5a961f13 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:16.870215 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4ba4436d-3696-403f-94ab-7a1f5a961f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:16.930309 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1442/2892] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:16.940497 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:16.942001 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:17.001029 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-143d18b0-eee1-4803-b589-09b421d90c61 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:17.005035 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabbc14-095f-48a6-93e9-942f1e93d859 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:17.005716 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabbc14-095f-48a6-93e9-942f1e93d859 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:17.005996 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabbc14-095f-48a6-93e9-942f1e93d859 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:17.006267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabbc14-095f-48a6-93e9-942f1e93d859 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:17.016328 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabbc14-095f-48a6-93e9-942f1e93d859 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:17.017795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabbc14-095f-48a6-93e9-942f1e93d859 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:17.027765 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1451/2893] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:17.301919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4436d-3696-403f-94ab-7a1f5a961f13 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:17.303425 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4ba4436d-3696-403f-94ab-7a1f5a961f13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:17.317754 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba4436d-3696-403f-94ab-7a1f5a961f13 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:17.322586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:17.323460 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:17.323881 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:17.324168 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] PATH_INFO: `/v3/projects/27f326d58b3d448885e835629d352dc1` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:17.325525 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f326d58b3d448885e835629d352dc1)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:17.327395 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:17.377642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f326d58b3d448885e835629d352dc1', 'request_id': 'req-70b0965a-b044-40a5-8a5b-6b28eba360db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.382946 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f326d58b3d448885e835629d352dc1', 'request_id': 'req-70b0965a-b044-40a5-8a5b-6b28eba360db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.390011 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f326d58b3d448885e835629d352dc1', 'request_id': 'req-70b0965a-b044-40a5-8a5b-6b28eba360db'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.401074 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] The token cache is being invalidate because project 27f326d58b3d448885e835629d352dc1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:17.401361 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.402592 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.404267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1443/2894] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:17 2022] DELETE /identity/v3/projects/27f326d58b3d448885e835629d352dc1 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:17.408257 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabbc14-095f-48a6-93e9-942f1e93d859 service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:17.420217 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabbc14-095f-48a6-93e9-942f1e93d859 service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:17.446702 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:17.478866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:17.479715 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:17.479993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:17.480285 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] PATH_INFO: `/v3/users/4a09e275eb5d455481034b534e6cc2a6` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:17.482148 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a09e275eb5d455481034b534e6cc2a6)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:17.484352 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:17.530527 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a09e275eb5d455481034b534e6cc2a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.542704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a09e275eb5d455481034b534e6cc2a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.550150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a09e275eb5d455481034b534e6cc2a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.550882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a09e275eb5d455481034b534e6cc2a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.551491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a09e275eb5d455481034b534e6cc2a6'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.564485 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1452/2895] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:17 2022] DELETE /identity/v3/users/4a09e275eb5d455481034b534e6cc2a6 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:17.835804 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:17.837828 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:17.853686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70b0965a-b044-40a5-8a5b-6b28eba360db admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:17.914639 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:17.915414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:17.915694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:17.915969 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] PATH_INFO: `/v3/projects/0180664745684a04ae7bab943d601f18` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:17.917342 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0180664745684a04ae7bab943d601f18)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:17.919603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:17.921114 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:17.975257 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0180664745684a04ae7bab943d601f18', 'request_id': 'req-204112b8-911d-4eb2-abf4-dfb6597410f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.981603 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0180664745684a04ae7bab943d601f18', 'request_id': 'req-204112b8-911d-4eb2-abf4-dfb6597410f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:17.988948 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0180664745684a04ae7bab943d601f18', 'request_id': 'req-204112b8-911d-4eb2-abf4-dfb6597410f0'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.002863 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] The token cache is being invalidate because project 0180664745684a04ae7bab943d601f18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:18.003158 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.003844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.005427 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1444/2896] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:17 2022] DELETE /identity/v3/projects/0180664745684a04ae7bab943d601f18 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:18.009725 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:18.011133 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:18.023841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1763bfee-295f-4c1d-a9f5-e640b6e1ea1b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:18.082334 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:18.083213 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:18.083498 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:18.083775 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] PATH_INFO: `/v3/users/11e1020dc82f4b7d91da386fc03eff92` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:18.084963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] RBAC: Authorizing `identity:delete_user(user_id=11e1020dc82f4b7d91da386fc03eff92)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:18.086915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:18.135101 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11e1020dc82f4b7d91da386fc03eff92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.137006 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=82,keystone:DELETE=17,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:18.150185 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11e1020dc82f4b7d91da386fc03eff92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.157589 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11e1020dc82f4b7d91da386fc03eff92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.158276 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11e1020dc82f4b7d91da386fc03eff92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.158877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-a9eccb9b-da14-4fbf-b7ba-df13a1c00497 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11e1020dc82f4b7d91da386fc03eff92'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.171883 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1453/2897] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:18 2022] DELETE /identity/v3/users/11e1020dc82f4b7d91da386fc03eff92 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:18.457530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:18.459023 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:18.473670 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-204112b8-911d-4eb2-abf4-dfb6597410f0 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:18.530865 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:18.531727 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:18.532014 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:18.532291 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] PATH_INFO: `/v3/projects/582eec9e1a1d4f169ba52a2426609ba4` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:18.533465 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] RBAC: Authorizing `identity:delete_project(project_id=582eec9e1a1d4f169ba52a2426609ba4)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:18.535428 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:18.580106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '582eec9e1a1d4f169ba52a2426609ba4', 'request_id': 'req-614d6ac7-f3c0-44ae-9708-807ea3f72230'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.585404 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '582eec9e1a1d4f169ba52a2426609ba4', 'request_id': 'req-614d6ac7-f3c0-44ae-9708-807ea3f72230'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.593031 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '582eec9e1a1d4f169ba52a2426609ba4', 'request_id': 'req-614d6ac7-f3c0-44ae-9708-807ea3f72230'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.604534 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] The token cache is being invalidate because project 582eec9e1a1d4f169ba52a2426609ba4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:18.605080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.606227 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-614d6ac7-f3c0-44ae-9708-807ea3f72230 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:18.608174 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1445/2898] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:18 2022] DELETE /identity/v3/projects/582eec9e1a1d4f169ba52a2426609ba4 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:18.613467 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716074ba-ff2a-4873-8a1c-6ed47d0ecd5f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:18.613779 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716074ba-ff2a-4873-8a1c-6ed47d0ecd5f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:18.614058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716074ba-ff2a-4873-8a1c-6ed47d0ecd5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:18.632071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-716074ba-ff2a-4873-8a1c-6ed47d0ecd5f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:18.633275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-716074ba-ff2a-4873-8a1c-6ed47d0ecd5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:18.694082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1454/2899] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:18.701823 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615d12c-6a99-465a-af3f-ed738418d6c2 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:18.702125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615d12c-6a99-465a-af3f-ed738418d6c2 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:18.702420 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615d12c-6a99-465a-af3f-ed738418d6c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:18.720642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-4615d12c-6a99-465a-af3f-ed738418d6c2 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:18.721889 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-4615d12c-6a99-465a-af3f-ed738418d6c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:18.729796 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1446/2900] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:18.734672 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716074ba-ff2a-4873-8a1c-6ed47d0ecd5f None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:18.744345 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716074ba-ff2a-4873-8a1c-6ed47d0ecd5f None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:18.751657 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615e636-0d53-4480-8452-401905de369b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:18.752327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615e636-0d53-4480-8452-401905de369b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:18.752612 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615e636-0d53-4480-8452-401905de369b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:18.752890 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4615e636-0d53-4480-8452-401905de369b admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:18.753664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615e636-0d53-4480-8452-401905de369b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:11:18.753979 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615e636-0d53-4480-8452-401905de369b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:18.755613 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4615e636-0d53-4480-8452-401905de369b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:18.768202 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1455/2901] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:11:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:11:18.773061 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615d12c-6a99-465a-af3f-ed738418d6c2 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:18.782640 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615d12c-6a99-465a-af3f-ed738418d6c2 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:18.787301 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8a24c-5ef6-428a-a267-a5c4dc9b6443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:18.787957 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8a24c-5ef6-428a-a267-a5c4dc9b6443 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:18.788244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8a24c-5ef6-428a-a267-a5c4dc9b6443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:18.788523 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e8a24c-5ef6-428a-a267-a5c4dc9b6443 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:18.789221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8a24c-5ef6-428a-a267-a5c4dc9b6443 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:18.791149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e8a24c-5ef6-428a-a267-a5c4dc9b6443 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:18.801541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:18.821440 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1447/2902] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:18 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:18.823993 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615e636-0d53-4480-8452-401905de369b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:18.834259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4615e636-0d53-4480-8452-401905de369b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:18.892949 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73c6a3-40da-4036-88fd-0d0d9f8a3894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:18.893639 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73c6a3-40da-4036-88fd-0d0d9f8a3894 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:18.893970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73c6a3-40da-4036-88fd-0d0d9f8a3894 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:18.894249 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c73c6a3-40da-4036-88fd-0d0d9f8a3894 admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:18.894939 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73c6a3-40da-4036-88fd-0d0d9f8a3894 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:18.896855 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c73c6a3-40da-4036-88fd-0d0d9f8a3894 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:18.920023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1456/2903] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:18 2022] POST /identity/v3/users => generated 296 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:18.922722 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8a24c-5ef6-428a-a267-a5c4dc9b6443 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:18.935071 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e8a24c-5ef6-428a-a267-a5c4dc9b6443 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:18.939985 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf5da7-f1d5-4067-b45f-90c073cbfde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:18.940732 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5da7-f1d5-4067-b45f-90c073cbfde8 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:18.941017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5da7-f1d5-4067-b45f-90c073cbfde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:18.941297 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeaf5da7-f1d5-4067-b45f-90c073cbfde8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:18.942230 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf5da7-f1d5-4067-b45f-90c073cbfde8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:18.943822 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeaf5da7-f1d5-4067-b45f-90c073cbfde8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:18.955053 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1448/2904] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:11:18.959092 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73c6a3-40da-4036-88fd-0d0d9f8a3894 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:18.970049 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c73c6a3-40da-4036-88fd-0d0d9f8a3894 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:18.975082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac1846-d687-4e41-a859-300d87ab359c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:18.975751 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac1846-d687-4e41-a859-300d87ab359c admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:18.976036 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac1846-d687-4e41-a859-300d87ab359c admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:18.976321 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac1846-d687-4e41-a859-300d87ab359c admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:18.977052 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac1846-d687-4e41-a859-300d87ab359c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:18.978682 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac1846-d687-4e41-a859-300d87ab359c admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:18.990407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1457/2905] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:18.994621 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf5da7-f1d5-4067-b45f-90c073cbfde8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:19.004611 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeaf5da7-f1d5-4067-b45f-90c073cbfde8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:19.009159 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5159496-4f06-4808-b8c7-40899e8b03a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:19.010066 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5159496-4f06-4808-b8c7-40899e8b03a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:19.010354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5159496-4f06-4808-b8c7-40899e8b03a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:19.010626 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5159496-4f06-4808-b8c7-40899e8b03a8 admin admin] PATH_INFO: `/v3/projects/9eb33e71a90f497b839e1a069a4548cf/users/1f0a379b53a444dda7d1fe8860b53aa2/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:19.025474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5159496-4f06-4808-b8c7-40899e8b03a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9eb33e71a90f497b839e1a069a4548cf, user_id=1f0a379b53a444dda7d1fe8860b53aa2, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:19.028964 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5159496-4f06-4808-b8c7-40899e8b03a8 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:19.041641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1449/2906] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:18 2022] PUT /identity/v3/projects/9eb33e71a90f497b839e1a069a4548cf/users/1f0a379b53a444dda7d1fe8860b53aa2/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:19.052789 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc5615-336c-43de-a6af-66d9a7285e0d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:19.053332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc5615-336c-43de-a6af-66d9a7285e0d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:19.053332 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5fc5615-336c-43de-a6af-66d9a7285e0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:19.080351 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-d5fc5615-336c-43de-a6af-66d9a7285e0d None None] MFA Rules not processed for user `1f0a379b53a444dda7d1fe8860b53aa2`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:19.081621 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-d5fc5615-336c-43de-a6af-66d9a7285e0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:19.102442 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:19.147158 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1458/2907] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:19 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:19.158463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5159496-4f06-4808-b8c7-40899e8b03a8 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:19.160105 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-c5159496-4f06-4808-b8c7-40899e8b03a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:19.218132 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5159496-4f06-4808-b8c7-40899e8b03a8 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:19.221675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc8118a-f993-4601-b1ae-31949da97b3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:19.222327 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8118a-f993-4601-b1ae-31949da97b3d service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:19.222614 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8118a-f993-4601-b1ae-31949da97b3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:19.222895 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc8118a-f993-4601-b1ae-31949da97b3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:19.231831 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8118a-f993-4601-b1ae-31949da97b3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:19.233267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fc8118a-f993-4601-b1ae-31949da97b3d service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:19.242266 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1450/2908] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:19 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:29.144251 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=17 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:29.238323 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=44,keystone:INSERT=2 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:35.285488 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc5615-336c-43de-a6af-66d9a7285e0d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:35.351199 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5fc5615-336c-43de-a6af-66d9a7285e0d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:35.355676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:35.356843 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:35.357128 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:35.357403 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] PATH_INFO: `/v3/users/1f0a379b53a444dda7d1fe8860b53aa2` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:35.358598 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1f0a379b53a444dda7d1fe8860b53aa2)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:35.360787 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:35.406776 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1f0a379b53a444dda7d1fe8860b53aa2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:35.418308 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1f0a379b53a444dda7d1fe8860b53aa2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:35.425309 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1f0a379b53a444dda7d1fe8860b53aa2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:35.426043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1f0a379b53a444dda7d1fe8860b53aa2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:35.426661 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1f0a379b53a444dda7d1fe8860b53aa2'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:35.439966 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1459/2909] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:35 2022] DELETE /identity/v3/users/1f0a379b53a444dda7d1fe8860b53aa2 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:35.444537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537a74b-0f84-4bea-8f79-b9650c2df7ce None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:35.445026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537a74b-0f84-4bea-8f79-b9650c2df7ce None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:35.445125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f537a74b-0f84-4bea-8f79-b9650c2df7ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:35.462852 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-f537a74b-0f84-4bea-8f79-b9650c2df7ce None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:35.464047 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f537a74b-0f84-4bea-8f79-b9650c2df7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:35.522118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1451/2910] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:35.531834 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:35.533250 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:35.590074 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-517e857f-067b-43b6-a778-dc2857d49dc6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:35.593961 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ca7db-03be-4503-9c4f-269fb4083a7b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:35.594600 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca7db-03be-4503-9c4f-269fb4083a7b service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:35.594882 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca7db-03be-4503-9c4f-269fb4083a7b service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:35.595157 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ca7db-03be-4503-9c4f-269fb4083a7b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:35.604287 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ca7db-03be-4503-9c4f-269fb4083a7b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:35.605750 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ca7db-03be-4503-9c4f-269fb4083a7b service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:35.615656 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1460/2911] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:35.885021 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537a74b-0f84-4bea-8f79-b9650c2df7ce None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:35.886538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-f537a74b-0f84-4bea-8f79-b9650c2df7ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:35.901625 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f537a74b-0f84-4bea-8f79-b9650c2df7ce None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:35.906319 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:35.907191 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:35.907473 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:35.907871 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] PATH_INFO: `/v3/projects/9eb33e71a90f497b839e1a069a4548cf` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:35.909282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] RBAC: Authorizing `identity:delete_project(project_id=9eb33e71a90f497b839e1a069a4548cf)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:35.911087 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:35.958134 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9eb33e71a90f497b839e1a069a4548cf', 'request_id': 'req-be418515-e28a-4381-a688-ba55e236cdfe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:35.963532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9eb33e71a90f497b839e1a069a4548cf', 'request_id': 'req-be418515-e28a-4381-a688-ba55e236cdfe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:35.970397 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9eb33e71a90f497b839e1a069a4548cf', 'request_id': 'req-be418515-e28a-4381-a688-ba55e236cdfe'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:35.982049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] The token cache is being invalidate because project 9eb33e71a90f497b839e1a069a4548cf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:35.982345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:35.982980 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-be418515-e28a-4381-a688-ba55e236cdfe admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:35.984520 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1452/2912] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:35 2022] DELETE /identity/v3/projects/9eb33e71a90f497b839e1a069a4548cf => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:35.991641 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bf03d-91db-43fe-a57b-c455388e4208 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:35.991931 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bf03d-91db-43fe-a57b-c455388e4208 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:35.992243 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd1bf03d-91db-43fe-a57b-c455388e4208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:36.010912 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-bd1bf03d-91db-43fe-a57b-c455388e4208 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:36.012109 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-bd1bf03d-91db-43fe-a57b-c455388e4208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:36.029298 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:36.074006 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1461/2913] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:36.082380 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9a7ff-4e4b-4751-a30b-556fa0b455b5 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:36.082709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9a7ff-4e4b-4751-a30b-556fa0b455b5 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:36.083062 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc9a7ff-4e4b-4751-a30b-556fa0b455b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:36.101799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-bcc9a7ff-4e4b-4751-a30b-556fa0b455b5 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:36.103027 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bcc9a7ff-4e4b-4751-a30b-556fa0b455b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:36.112171 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1453/2914] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:36.116236 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bf03d-91db-43fe-a57b-c455388e4208 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:36.125896 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd1bf03d-91db-43fe-a57b-c455388e4208 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:36.130412 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f2b89-40b3-472a-b56b-46632afe7605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:36.131054 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f2b89-40b3-472a-b56b-46632afe7605 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:36.131335 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f2b89-40b3-472a-b56b-46632afe7605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:36.131608 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f2b89-40b3-472a-b56b-46632afe7605 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:36.132320 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f2b89-40b3-472a-b56b-46632afe7605 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:11:36.132625 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f2b89-40b3-472a-b56b-46632afe7605 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:36.134262 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3f2b89-40b3-472a-b56b-46632afe7605 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:36.145825 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1462/2915] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:11:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 20 20:11:36.149660 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9a7ff-4e4b-4751-a30b-556fa0b455b5 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:36.159295 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc9a7ff-4e4b-4751-a30b-556fa0b455b5 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:36.163915 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783738ec-a247-4c70-b3ad-4227805eeb44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:36.164563 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783738ec-a247-4c70-b3ad-4227805eeb44 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:36.164849 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783738ec-a247-4c70-b3ad-4227805eeb44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:36.165125 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-783738ec-a247-4c70-b3ad-4227805eeb44 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:36.165844 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783738ec-a247-4c70-b3ad-4227805eeb44 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:36.167750 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-783738ec-a247-4c70-b3ad-4227805eeb44 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:36.193088 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1454/2916] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:36 2022] POST /identity/v3/projects => generated 385 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:36.197210 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f2b89-40b3-472a-b56b-46632afe7605 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:36.207267 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3f2b89-40b3-472a-b56b-46632afe7605 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:36.263429 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f12f8-80d6-45fe-9898-2afaa29ad92d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:36.264088 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f12f8-80d6-45fe-9898-2afaa29ad92d admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:36.264375 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f12f8-80d6-45fe-9898-2afaa29ad92d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:36.264655 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804f12f8-80d6-45fe-9898-2afaa29ad92d admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:36.265349 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f12f8-80d6-45fe-9898-2afaa29ad92d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:36.267333 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804f12f8-80d6-45fe-9898-2afaa29ad92d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:36.287432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1463/2917] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:36 2022] POST /identity/v3/users => generated 293 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:11:36.291412 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783738ec-a247-4c70-b3ad-4227805eeb44 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:36.302084 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-783738ec-a247-4c70-b3ad-4227805eeb44 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:36.307481 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353a0db-eb75-41ec-850f-dcc3ebcd2de6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:36.308275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353a0db-eb75-41ec-850f-dcc3ebcd2de6 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:36.308810 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353a0db-eb75-41ec-850f-dcc3ebcd2de6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:36.309104 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8353a0db-eb75-41ec-850f-dcc3ebcd2de6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:36.310054 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353a0db-eb75-41ec-850f-dcc3ebcd2de6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:36.311698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8353a0db-eb75-41ec-850f-dcc3ebcd2de6 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:36.322554 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1455/2918] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:11:36.326512 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f12f8-80d6-45fe-9898-2afaa29ad92d admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:36.336539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804f12f8-80d6-45fe-9898-2afaa29ad92d admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:36.341619 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfabdbc-69ef-40dc-91f6-709f78757fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:36.342269 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfabdbc-69ef-40dc-91f6-709f78757fcc admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:36.342550 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfabdbc-69ef-40dc-91f6-709f78757fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:36.342823 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfabdbc-69ef-40dc-91f6-709f78757fcc admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:36.343545 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfabdbc-69ef-40dc-91f6-709f78757fcc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:36.345126 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfabdbc-69ef-40dc-91f6-709f78757fcc admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:36.354929 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1464/2919] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:36 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:36.359584 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353a0db-eb75-41ec-850f-dcc3ebcd2de6 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:36.370025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8353a0db-eb75-41ec-850f-dcc3ebcd2de6 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:36.375078 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd948e10-196d-4e33-9d76-94f5a3ad023d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:36.375978 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd948e10-196d-4e33-9d76-94f5a3ad023d admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:36.376268 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd948e10-196d-4e33-9d76-94f5a3ad023d admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:36.376635 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd948e10-196d-4e33-9d76-94f5a3ad023d admin admin] PATH_INFO: `/v3/projects/cc8b7ce45a7d420c8e80648aef1eb14f/users/e08007dee07e4884b37e4beb78956be5/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:36.392345 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd948e10-196d-4e33-9d76-94f5a3ad023d admin admin] RBAC: Authorizing `identity:create_grant(project_id=cc8b7ce45a7d420c8e80648aef1eb14f, user_id=e08007dee07e4884b37e4beb78956be5, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:36.395075 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd948e10-196d-4e33-9d76-94f5a3ad023d admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:36.408162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1456/2920] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:36 2022] PUT /identity/v3/projects/cc8b7ce45a7d420c8e80648aef1eb14f/users/e08007dee07e4884b37e4beb78956be5/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:36.413160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ca8d9-65f8-4bb2-bbcc-2c86485967cd None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:36.413444 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ca8d9-65f8-4bb2-bbcc-2c86485967cd None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:36.413752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206ca8d9-65f8-4bb2-bbcc-2c86485967cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:36.432167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-206ca8d9-65f8-4bb2-bbcc-2c86485967cd None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:36.433392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-206ca8d9-65f8-4bb2-bbcc-2c86485967cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:36.491407 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1465/2921] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:36.501149 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd948e10-196d-4e33-9d76-94f5a3ad023d admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:36.502953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-cd948e10-196d-4e33-9d76-94f5a3ad023d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:36.524779 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:36.562082 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd948e10-196d-4e33-9d76-94f5a3ad023d admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:36.565620 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6802836f-d75e-492f-be53-5b4a47502bd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:36.566308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802836f-d75e-492f-be53-5b4a47502bd7 service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:36.566589 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802836f-d75e-492f-be53-5b4a47502bd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:36.566864 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6802836f-d75e-492f-be53-5b4a47502bd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:36.576106 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802836f-d75e-492f-be53-5b4a47502bd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:36.577631 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6802836f-d75e-492f-be53-5b4a47502bd7 service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:36.586847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1457/2922] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:39.321534 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ca8d9-65f8-4bb2-bbcc-2c86485967cd None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:39.323510 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-206ca8d9-65f8-4bb2-bbcc-2c86485967cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:39.345275 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-206ca8d9-65f8-4bb2-bbcc-2c86485967cd None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:39.350410 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:39.351387 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:39.351742 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:39.352090 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] PATH_INFO: `/v3/users/04d8e0ba1a284b98a667324dc3900c87` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:39.353719 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04d8e0ba1a284b98a667324dc3900c87)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:39.358677 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:39.405432 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:39.450169 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04d8e0ba1a284b98a667324dc3900c87'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:39.475473 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04d8e0ba1a284b98a667324dc3900c87'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:39.485487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04d8e0ba1a284b98a667324dc3900c87'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:39.486318 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04d8e0ba1a284b98a667324dc3900c87'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:39.486970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04d8e0ba1a284b98a667324dc3900c87'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:39.503048 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1466/2923] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:39 2022] DELETE /identity/v3/users/04d8e0ba1a284b98a667324dc3900c87 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:39.507986 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec26b7-2e90-445a-9032-60860086c86f None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:39.508492 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec26b7-2e90-445a-9032-60860086c86f None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:39.508593 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec26b7-2e90-445a-9032-60860086c86f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:39.543140 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-8bec26b7-2e90-445a-9032-60860086c86f None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:39.547698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8bec26b7-2e90-445a-9032-60860086c86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:39.619491 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1458/2924] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:39.629472 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:39.631433 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:39.755770 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7e4f968-5be3-4f7d-affb-bfeb236067e3 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:39.768586 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e12d851-fd25-4d09-8d25-c073e00d140e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:39.769573 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12d851-fd25-4d09-8d25-c073e00d140e service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:39.770031 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12d851-fd25-4d09-8d25-c073e00d140e service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:39.771090 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e12d851-fd25-4d09-8d25-c073e00d140e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:39.788989 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12d851-fd25-4d09-8d25-c073e00d140e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:39.792160 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e12d851-fd25-4d09-8d25-c073e00d140e service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:39.821218 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1467/2925] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:40.278547 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec26b7-2e90-445a-9032-60860086c86f None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:40.280212 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-8bec26b7-2e90-445a-9032-60860086c86f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:40.301731 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bec26b7-2e90-445a-9032-60860086c86f None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:40.330449 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:40.333024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.333574 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.334058 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] PATH_INFO: `/v3/projects/8550e436c13f43d895ad4934a3ab2503` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.335990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=8550e436c13f43d895ad4934a3ab2503)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:40.338194 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:40.413954 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8550e436c13f43d895ad4934a3ab2503', 'request_id': 'req-122e5cc0-55e9-4958-a85b-0c2e153024cf'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:40.425463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8550e436c13f43d895ad4934a3ab2503', 'request_id': 'req-122e5cc0-55e9-4958-a85b-0c2e153024cf'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:40.444530 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8550e436c13f43d895ad4934a3ab2503', 'request_id': 'req-122e5cc0-55e9-4958-a85b-0c2e153024cf'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:40.459542 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] The token cache is being invalidate because project 8550e436c13f43d895ad4934a3ab2503 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:40.459847 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:40.460475 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-122e5cc0-55e9-4958-a85b-0c2e153024cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:40.462459 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1459/2926] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:40 2022] DELETE /identity/v3/projects/8550e436c13f43d895ad4934a3ab2503 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:40.468156 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae518c-62d1-40e8-afaf-c5d2b2f688ec None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.468860 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae518c-62d1-40e8-afaf-c5d2b2f688ec None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.469645 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ae518c-62d1-40e8-afaf-c5d2b2f688ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.493630 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-a1ae518c-62d1-40e8-afaf-c5d2b2f688ec None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:40.495146 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-a1ae518c-62d1-40e8-afaf-c5d2b2f688ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:40.550934 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae9fe9-2fdc-40de-b6c0-de5170d0b834 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.552090 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae9fe9-2fdc-40de-b6c0-de5170d0b834 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.552533 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbae9fe9-2fdc-40de-b6c0-de5170d0b834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.586213 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-fbae9fe9-2fdc-40de-b6c0-de5170d0b834 None None] MFA Rules not processed for user `e08007dee07e4884b37e4beb78956be5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:40.587419 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fbae9fe9-2fdc-40de-b6c0-de5170d0b834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:40.592428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1468/2927] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:40.600696 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d21d7-b3ef-4f6a-9d9e-40b5ee91de5d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.600897 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d21d7-b3ef-4f6a-9d9e-40b5ee91de5d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.601017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254d21d7-b3ef-4f6a-9d9e-40b5ee91de5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.620704 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-254d21d7-b3ef-4f6a-9d9e-40b5ee91de5d None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:40.621974 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-254d21d7-b3ef-4f6a-9d9e-40b5ee91de5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:40.631023 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1469/2928] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:40.635406 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d21d7-b3ef-4f6a-9d9e-40b5ee91de5d None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:40.645647 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254d21d7-b3ef-4f6a-9d9e-40b5ee91de5d None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:40.647514 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:40.655828 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c164a03-9fde-43ed-90e0-4fe85c9cb6d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:40.656515 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c164a03-9fde-43ed-90e0-4fe85c9cb6d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.656762 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c164a03-9fde-43ed-90e0-4fe85c9cb6d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.657071 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c164a03-9fde-43ed-90e0-4fe85c9cb6d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.658007 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c164a03-9fde-43ed-90e0-4fe85c9cb6d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80210) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 20 20:11:40.658314 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c164a03-9fde-43ed-90e0-4fe85c9cb6d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:40.659915 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c164a03-9fde-43ed-90e0-4fe85c9cb6d9 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:40.673019 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1470/2929] 10.222.0.69 () {58 vars in 1152 bytes} [Wed Jul 20 20:11:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 20 20:11:40.677385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c164a03-9fde-43ed-90e0-4fe85c9cb6d9 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:40.692396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c164a03-9fde-43ed-90e0-4fe85c9cb6d9 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:40.698878 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e75da-8057-4cfe-901b-f414596fe0f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:40.699353 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e75da-8057-4cfe-901b-f414596fe0f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.699869 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e75da-8057-4cfe-901b-f414596fe0f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.700007 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1460/2930] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:40.702487 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b22e75da-8057-4cfe-901b-f414596fe0f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.703226 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e75da-8057-4cfe-901b-f414596fe0f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:40.704928 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae9fe9-2fdc-40de-b6c0-de5170d0b834 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:40.705446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b22e75da-8057-4cfe-901b-f414596fe0f8 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:40.706293 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fbae9fe9-2fdc-40de-b6c0-de5170d0b834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:40.719967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbae9fe9-2fdc-40de-b6c0-de5170d0b834 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:40.725108 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c29334-0239-4f00-baca-fb1c2e9e3675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:40.726057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29334-0239-4f00-baca-fb1c2e9e3675 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.726354 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29334-0239-4f00-baca-fb1c2e9e3675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.726641 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c29334-0239-4f00-baca-fb1c2e9e3675 admin admin] PATH_INFO: `/v3/projects` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.727328 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c29334-0239-4f00-baca-fb1c2e9e3675 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:40.729263 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87c29334-0239-4f00-baca-fb1c2e9e3675 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:40.737189 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1471/2931] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:40 2022] POST /identity/v3/projects => generated 385 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 20 20:11:40.741061 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e75da-8057-4cfe-901b-f414596fe0f8 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:40.762155 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=92,keystone:DELETE=6,keystone:INSERT=2 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:40.785914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1461/2932] 10.222.0.69 () {60 vars in 1152 bytes} [Wed Jul 20 20:11:40 2022] POST /identity/v3/projects => generated 385 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:11:40.788013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c29334-0239-4f00-baca-fb1c2e9e3675 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:40.840539 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b22e75da-8057-4cfe-901b-f414596fe0f8 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:40.864586 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c29334-0239-4f00-baca-fb1c2e9e3675 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:40.871782 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27e599-128b-4717-b220-ddd341674f27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:40.873270 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27e599-128b-4717-b220-ddd341674f27 admin admin] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.873686 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27e599-128b-4717-b220-ddd341674f27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.873974 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b27e599-128b-4717-b220-ddd341674f27 admin admin] PATH_INFO: `/v3/users` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.874678 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27e599-128b-4717-b220-ddd341674f27 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:40.876963 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b27e599-128b-4717-b220-ddd341674f27 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:40.904920 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1462/2933] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:40 2022] POST /identity/v3/users => generated 299 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:11:40.908905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27e599-128b-4717-b220-ddd341674f27 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:40.912259 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50624a8e-cd19-4737-84b5-24e4cdeb7d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:40.912986 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50624a8e-cd19-4737-84b5-24e4cdeb7d2e admin admin] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.913273 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50624a8e-cd19-4737-84b5-24e4cdeb7d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.913583 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50624a8e-cd19-4737-84b5-24e4cdeb7d2e admin admin] PATH_INFO: `/v3/users` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.914535 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50624a8e-cd19-4737-84b5-24e4cdeb7d2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:40.917369 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50624a8e-cd19-4737-84b5-24e4cdeb7d2e admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:40.920387 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b27e599-128b-4717-b220-ddd341674f27 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:40.926153 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483c753-a14f-4229-a0a0-952190498602 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:40.927070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483c753-a14f-4229-a0a0-952190498602 admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.927538 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483c753-a14f-4229-a0a0-952190498602 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.927962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7483c753-a14f-4229-a0a0-952190498602 admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.928970 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483c753-a14f-4229-a0a0-952190498602 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:40.930694 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7483c753-a14f-4229-a0a0-952190498602 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:40.942520 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1472/2934] 10.222.0.69 () {60 vars in 1143 bytes} [Wed Jul 20 20:11:40 2022] POST /identity/v3/users => generated 293 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 20 20:11:40.943683 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1463/2935] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:11:40.946427 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50624a8e-cd19-4737-84b5-24e4cdeb7d2e admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:40.947652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483c753-a14f-4229-a0a0-952190498602 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:40.958069 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50624a8e-cd19-4737-84b5-24e4cdeb7d2e admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:40.958267 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7483c753-a14f-4229-a0a0-952190498602 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:40.963063 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af0bc3-2c8b-4e05-9746-75df11d36d27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:40.963794 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9bc69-f2e6-46a3-8c89-e5a977d0d38b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:40.964002 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af0bc3-2c8b-4e05-9746-75df11d36d27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.964284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af0bc3-2c8b-4e05-9746-75df11d36d27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.964454 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9bc69-f2e6-46a3-8c89-e5a977d0d38b admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:40.964550 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56af0bc3-2c8b-4e05-9746-75df11d36d27 admin admin] PATH_INFO: `/v3/projects/5ef3a9afb43547c383faad5d05efd1d7/users/96be00144de941169cc676efcfd16736/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.964737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9bc69-f2e6-46a3-8c89-e5a977d0d38b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:40.965017 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8b9bc69-f2e6-46a3-8c89-e5a977d0d38b admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:40.966574 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9bc69-f2e6-46a3-8c89-e5a977d0d38b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:40.968122 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8b9bc69-f2e6-46a3-8c89-e5a977d0d38b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:40.982804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1473/2936] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:11:40.983246 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af0bc3-2c8b-4e05-9746-75df11d36d27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ef3a9afb43547c383faad5d05efd1d7, user_id=96be00144de941169cc676efcfd16736, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:40.983877 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9bc69-f2e6-46a3-8c89-e5a977d0d38b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:40.986242 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56af0bc3-2c8b-4e05-9746-75df11d36d27 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:40.995440 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8b9bc69-f2e6-46a3-8c89-e5a977d0d38b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:41.002972 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1464/2937] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:40 2022] PUT /identity/v3/projects/5ef3a9afb43547c383faad5d05efd1d7/users/96be00144de941169cc676efcfd16736/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:41.006880 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af0bc3-2c8b-4e05-9746-75df11d36d27 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:41.060329 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397f1cb-1ec4-48b2-a715-944ed0cbde0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:41.061449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397f1cb-1ec4-48b2-a715-944ed0cbde0a admin admin] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:41.062428 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397f1cb-1ec4-48b2-a715-944ed0cbde0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:41.062778 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1397f1cb-1ec4-48b2-a715-944ed0cbde0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:41.063766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397f1cb-1ec4-48b2-a715-944ed0cbde0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:41.066188 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1397f1cb-1ec4-48b2-a715-944ed0cbde0a admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:41.078952 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1474/2938] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:40 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 20 20:11:41.083181 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397f1cb-1ec4-48b2-a715-944ed0cbde0a admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:41.093463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56af0bc3-2c8b-4e05-9746-75df11d36d27 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:41.094654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1397f1cb-1ec4-48b2-a715-944ed0cbde0a admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:41.098612 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f270ff3-ad8e-4965-b270-cfd659cfb1dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:41.099368 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f135487-653e-499b-bbb4-f1143961039d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:41.099532 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f270ff3-ad8e-4965-b270-cfd659cfb1dc admin admin] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:41.099813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f270ff3-ad8e-4965-b270-cfd659cfb1dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:41.100162 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f270ff3-ad8e-4965-b270-cfd659cfb1dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:41.100492 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f135487-653e-499b-bbb4-f1143961039d admin admin] REQUEST_METHOD: `PUT` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:41.100841 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f135487-653e-499b-bbb4-f1143961039d admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:41.101176 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f135487-653e-499b-bbb4-f1143961039d admin admin] PATH_INFO: `/v3/projects/708846cc874b43378cedd2e92b44892d/users/84c9846bad014d0eba063986bd4e570e/roles/91f9d5c6716547c18da989bf327f6aca` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:41.101274 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f270ff3-ad8e-4965-b270-cfd659cfb1dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:41.102990 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f270ff3-ad8e-4965-b270-cfd659cfb1dc admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:41.114259 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1465/2939] 10.222.0.69 () {58 vars in 1121 bytes} [Wed Jul 20 20:11:41 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 20 20:11:41.118308 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f270ff3-ad8e-4965-b270-cfd659cfb1dc admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:41.124341 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f135487-653e-499b-bbb4-f1143961039d admin admin] RBAC: Authorizing `identity:create_grant(project_id=708846cc874b43378cedd2e92b44892d, user_id=84c9846bad014d0eba063986bd4e570e, role_id=91f9d5c6716547c18da989bf327f6aca)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:41.125451 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:41.127377 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f135487-653e-499b-bbb4-f1143961039d admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:41.129813 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f270ff3-ad8e-4965-b270-cfd659cfb1dc admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:41.136167 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a049de-5ff7-4d78-9683-0fdce8127050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:41.137352 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a049de-5ff7-4d78-9683-0fdce8127050 admin admin] REQUEST_METHOD: `PUT` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:41.138483 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a049de-5ff7-4d78-9683-0fdce8127050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:41.139184 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:41.139672 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5a049de-5ff7-4d78-9683-0fdce8127050 admin admin] PATH_INFO: `/v3/projects/5ef3a9afb43547c383faad5d05efd1d7/users/96be00144de941169cc676efcfd16736/roles/58ca114197894385aca5fa8e7cf9f4c7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:41.142024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a049de-5ff7-4d78-9683-0fdce8127050 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5ef3a9afb43547c383faad5d05efd1d7, user_id=96be00144de941169cc676efcfd16736, role_id=58ca114197894385aca5fa8e7cf9f4c7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:41.145239 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5a049de-5ff7-4d78-9683-0fdce8127050 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:41.145997 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1475/2940] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:41 2022] PUT /identity/v3/projects/708846cc874b43378cedd2e92b44892d/users/84c9846bad014d0eba063986bd4e570e/roles/91f9d5c6716547c18da989bf327f6aca => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:41.156950 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524a282-993a-405a-9fe4-7b4f87bb786d None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:41.157246 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524a282-993a-405a-9fe4-7b4f87bb786d None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:41.157558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524a282-993a-405a-9fe4-7b4f87bb786d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:41.159962 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1466/2941] 10.222.0.69 () {60 vars in 1482 bytes} [Wed Jul 20 20:11:41 2022] PUT /identity/v3/projects/5ef3a9afb43547c383faad5d05efd1d7/users/96be00144de941169cc676efcfd16736/roles/58ca114197894385aca5fa8e7cf9f4c7 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 20 20:11:41.191265 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-5524a282-993a-405a-9fe4-7b4f87bb786d None None] MFA Rules not processed for user `84c9846bad014d0eba063986bd4e570e`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:41.195401 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-5524a282-993a-405a-9fe4-7b4f87bb786d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:41.307077 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1476/2942] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:41.321214 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a049de-5ff7-4d78-9683-0fdce8127050 admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:41.322675 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b5a049de-5ff7-4d78-9683-0fdce8127050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:41.404118 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5a049de-5ff7-4d78-9683-0fdce8127050 admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:41.410302 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e2e77-5af0-4420-bfdc-affca3fbae48 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c94d9030f61a4e1f8b3d75ef8925bdd0', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:41.411184 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e2e77-5af0-4420-bfdc-affca3fbae48 service cinder] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:41.411898 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e2e77-5af0-4420-bfdc-affca3fbae48 service cinder] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:41.412183 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731e2e77-5af0-4420-bfdc-affca3fbae48 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:41.423792 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e2e77-5af0-4420-bfdc-affca3fbae48 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:41.425709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-731e2e77-5af0-4420-bfdc-affca3fbae48 service cinder] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:41.437489 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1467/2943] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:45.730767 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeebde3-adbc-44eb-a41f-78fcd9c5150f None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:45.731303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeebde3-adbc-44eb-a41f-78fcd9c5150f None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:45.731303 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eeebde3-adbc-44eb-a41f-78fcd9c5150f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:45.759322 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-4eeebde3-adbc-44eb-a41f-78fcd9c5150f None None] MFA Rules not processed for user `96be00144de941169cc676efcfd16736`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:45.760525 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-4eeebde3-adbc-44eb-a41f-78fcd9c5150f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:45.847752 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1477/2944] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:45 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:45.864414 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e2e77-5af0-4420-bfdc-affca3fbae48 service cinder] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:45.866037 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-731e2e77-5af0-4420-bfdc-affca3fbae48 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:45.929757 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731e2e77-5af0-4420-bfdc-affca3fbae48 service cinder] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:45.934049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f591b-dedd-46dd-9006-2ec6bd88574c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3e46ec9c31094d03b6a413e09bc9f699', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:45.934716 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f591b-dedd-46dd-9006-2ec6bd88574c service glance] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:45.935206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f591b-dedd-46dd-9006-2ec6bd88574c service glance] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:45.935541 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3f591b-dedd-46dd-9006-2ec6bd88574c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:45.994876 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f591b-dedd-46dd-9006-2ec6bd88574c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:45.996344 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3f591b-dedd-46dd-9006-2ec6bd88574c service glance] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:46.005285 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1468/2945] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:45 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:11:46.105381 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b1e66-0016-4b9d-abd5-6df4ba5d89b6 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:46.105688 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b1e66-0016-4b9d-abd5-6df4ba5d89b6 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:46.105970 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-743b1e66-0016-4b9d-abd5-6df4ba5d89b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:46.126663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-743b1e66-0016-4b9d-abd5-6df4ba5d89b6 None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:46.127906 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-743b1e66-0016-4b9d-abd5-6df4ba5d89b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:46.187873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1478/2946] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 20:11:46 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:46.240131 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f591b-dedd-46dd-9006-2ec6bd88574c service glance] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:46.241753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-fa3f591b-dedd-46dd-9006-2ec6bd88574c service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:46.299511 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:46.336080 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3f591b-dedd-46dd-9006-2ec6bd88574c service glance] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:46.345743 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e07658b-6076-4dbc-848b-726a2c80a40c service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:46.346453 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07658b-6076-4dbc-848b-726a2c80a40c service swift] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:46.346744 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07658b-6076-4dbc-848b-726a2c80a40c service swift] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:46.347025 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e07658b-6076-4dbc-848b-726a2c80a40c service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:46.356932 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07658b-6076-4dbc-848b-726a2c80a40c service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:46.359280 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e07658b-6076-4dbc-848b-726a2c80a40c service swift] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:46.373854 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1469/2947] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 20:11:46 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 20:11:50.479663 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b1e66-0016-4b9d-abd5-6df4ba5d89b6 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:50.522505 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=100 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:50.608451 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-743b1e66-0016-4b9d-abd5-6df4ba5d89b6 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:50.627935 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:50.628944 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:50.629310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:50.629698 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] PATH_INFO: `/v3/users/84c9846bad014d0eba063986bd4e570e` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:50.631076 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84c9846bad014d0eba063986bd4e570e)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:50.634043 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:50.709766 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84c9846bad014d0eba063986bd4e570e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:50.729817 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84c9846bad014d0eba063986bd4e570e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:50.744183 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84c9846bad014d0eba063986bd4e570e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:50.745253 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84c9846bad014d0eba063986bd4e570e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:50.745921 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84c9846bad014d0eba063986bd4e570e'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:50.768664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1479/2948] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:11:50 2022] DELETE /identity/v3/users/84c9846bad014d0eba063986bd4e570e => generated 0 bytes in 290 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:50.774013 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9985-00fe-4c2d-a15c-1fb6b1825034 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:50.774472 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9985-00fe-4c2d-a15c-1fb6b1825034 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:50.774811 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7b9985-00fe-4c2d-a15c-1fb6b1825034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:50.795010 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-de7b9985-00fe-4c2d-a15c-1fb6b1825034 None None] MFA Rules not processed for user `c9cf50bf3f924b6dac930c74893f5fb5`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:50.796217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-de7b9985-00fe-4c2d-a15c-1fb6b1825034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:50.889070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1470/2949] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:50.889956 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:50.891491 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:50.980623 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4f2e9b-c9fd-48b4-8bdf-c04a2bf8fab0 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:50.986998 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae33336b-6e00-4e76-921e-0869b1f49666 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:50.987670 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33336b-6e00-4e76-921e-0869b1f49666 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:50.987958 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33336b-6e00-4e76-921e-0869b1f49666 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:50.988384 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae33336b-6e00-4e76-921e-0869b1f49666 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:51.008824 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33336b-6e00-4e76-921e-0869b1f49666 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:51.011003 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae33336b-6e00-4e76-921e-0869b1f49666 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:51.030032 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1480/2950] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:11:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 20 20:11:51.443701 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b9985-00fe-4c2d-a15c-1fb6b1825034 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:51.445403 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-de7b9985-00fe-4c2d-a15c-1fb6b1825034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:51.469712 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7b9985-00fe-4c2d-a15c-1fb6b1825034 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:51.475196 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:51.476202 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:51.476580 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:51.476953 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] PATH_INFO: `/v3/projects/708846cc874b43378cedd2e92b44892d` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:51.479606 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=708846cc874b43378cedd2e92b44892d)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:51.482929 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:51.557979 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '708846cc874b43378cedd2e92b44892d', 'request_id': 'req-29d72dad-34bf-4a74-b12a-b3246f5229b7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:51.564799 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '708846cc874b43378cedd2e92b44892d', 'request_id': 'req-29d72dad-34bf-4a74-b12a-b3246f5229b7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:51.576826 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '708846cc874b43378cedd2e92b44892d', 'request_id': 'req-29d72dad-34bf-4a74-b12a-b3246f5229b7'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:51.597284 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] The token cache is being invalidate because project 708846cc874b43378cedd2e92b44892d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:11:51.597629 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:51.598277 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-29d72dad-34bf-4a74-b12a-b3246f5229b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:11:51.600097 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1471/2951] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:11:51 2022] DELETE /identity/v3/projects/708846cc874b43378cedd2e92b44892d => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:11:55.625191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca69e7-4d7f-4be6-850f-3f57b92bbb0a None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:55.625745 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca69e7-4d7f-4be6-850f-3f57b92bbb0a None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:55.625847 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ca69e7-4d7f-4be6-850f-3f57b92bbb0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:55.643892 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-69ca69e7-4d7f-4be6-850f-3f57b92bbb0a None None] MFA Rules not processed for user `e08007dee07e4884b37e4beb78956be5`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:55.645135 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-69ca69e7-4d7f-4be6-850f-3f57b92bbb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:55.683409 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:11:55.702856 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1481/2952] 10.222.0.69 () {58 vars in 957 bytes} [Wed Jul 20 20:11:55 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 20 20:11:57.416674 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27601751-6b7d-4149-b310-3743838777d9 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:57.417221 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27601751-6b7d-4149-b310-3743838777d9 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:57.417441 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27601751-6b7d-4149-b310-3743838777d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:57.438057 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-27601751-6b7d-4149-b310-3743838777d9 None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:11:57.439333 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-27601751-6b7d-4149-b310-3743838777d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:57.496967 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1472/2953] 10.222.0.69 () {58 vars in 1009 bytes} [Wed Jul 20 20:11:57 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:11:57.509700 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca69e7-4d7f-4be6-850f-3f57b92bbb0a None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:11:57.511150 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-69ca69e7-4d7f-4be6-850f-3f57b92bbb0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:11:57.582310 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ca69e7-4d7f-4be6-850f-3f57b92bbb0a None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:11:57.587327 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53f66f-eb52-49aa-9f12-15c51d3280c8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:11:57.587969 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53f66f-eb52-49aa-9f12-15c51d3280c8 service swift] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:11:57.588254 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53f66f-eb52-49aa-9f12-15c51d3280c8 service swift] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:11:57.588537 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d53f66f-eb52-49aa-9f12-15c51d3280c8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:11:57.596858 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53f66f-eb52-49aa-9f12-15c51d3280c8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:11:57.598462 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d53f66f-eb52-49aa-9f12-15c51d3280c8 service swift] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:11:57.607889 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1482/2954] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 20:11:57 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 20:12:00.868773 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27601751-6b7d-4149-b310-3743838777d9 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:12:00.870195 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-27601751-6b7d-4149-b310-3743838777d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:12:00.880786 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:12:00.932914 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27601751-6b7d-4149-b310-3743838777d9 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:12:00.937600 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b7e4a-3817-4bf9-9db0-98b2bebdcbbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:12:00.938307 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b7e4a-3817-4bf9-9db0-98b2bebdcbbb service neutron] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:12:00.938622 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b7e4a-3817-4bf9-9db0-98b2bebdcbbb service neutron] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:12:00.939016 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732b7e4a-3817-4bf9-9db0-98b2bebdcbbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:12:00.940976 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-732b7e4a-3817-4bf9-9db0-98b2bebdcbbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:12:01.037556 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b7e4a-3817-4bf9-9db0-98b2bebdcbbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:12:01.038992 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-732b7e4a-3817-4bf9-9db0-98b2bebdcbbb service neutron] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:12:01.052116 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1473/2955] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:12:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:12:07.603447 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=38 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:12:08.257178 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53f66f-eb52-49aa-9f12-15c51d3280c8 service swift] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:12:08.266797 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d53f66f-eb52-49aa-9f12-15c51d3280c8 service swift] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:12:08.270385 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff3c97-6864-4ef2-a952-07b2048d13ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:12:08.271012 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3c97-6864-4ef2-a952-07b2048d13ea service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:12:08.271295 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3c97-6864-4ef2-a952-07b2048d13ea service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:12:08.271570 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afff3c97-6864-4ef2-a952-07b2048d13ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:12:08.273331 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-afff3c97-6864-4ef2-a952-07b2048d13ea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:12:08.283459 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff3c97-6864-4ef2-a952-07b2048d13ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:12:08.284872 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afff3c97-6864-4ef2-a952-07b2048d13ea service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:12:08.293067 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1483/2956] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:12:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:12:11.047389 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=56 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:12:18.289338 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:12:24.269024 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b7e4a-3817-4bf9-9db0-98b2bebdcbbb service neutron] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:12:24.270537 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-732b7e4a-3817-4bf9-9db0-98b2bebdcbbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:12:24.379143 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732b7e4a-3817-4bf9-9db0-98b2bebdcbbb service neutron] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:12:24.384211 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc5617-42b1-4bad-a9a2-8b07b4ac0395 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:12:24.384861 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc5617-42b1-4bad-a9a2-8b07b4ac0395 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:12:24.385177 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc5617-42b1-4bad-a9a2-8b07b4ac0395 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:12:24.385456 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cdc5617-42b1-4bad-a9a2-8b07b4ac0395 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:12:24.387248 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1cdc5617-42b1-4bad-a9a2-8b07b4ac0395 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:12:24.398655 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc5617-42b1-4bad-a9a2-8b07b4ac0395 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:12:24.400240 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cdc5617-42b1-4bad-a9a2-8b07b4ac0395 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:12:24.410657 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1474/2957] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:12:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:12:30.234669 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff3c97-6864-4ef2-a952-07b2048d13ea service neutron] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:12:30.243609 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afff3c97-6864-4ef2-a952-07b2048d13ea service neutron] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:12:30.247232 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c4b88d-811c-4d52-9caa-d99d62615cd8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3a1a88089d384964bd9774eda4c5d8a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:12:30.247862 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4b88d-811c-4d52-9caa-d99d62615cd8 service neutron] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:12:30.248145 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4b88d-811c-4d52-9caa-d99d62615cd8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:12:30.248422 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c4b88d-811c-4d52-9caa-d99d62615cd8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:12:30.250191 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-79c4b88d-811c-4d52-9caa-d99d62615cd8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:12:30.321174 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4b88d-811c-4d52-9caa-d99d62615cd8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:12:30.322580 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79c4b88d-811c-4d52-9caa-d99d62615cd8 service neutron] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:12:30.332579 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1484/2958] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:12:30 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:12:34.406341 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=49 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:12:40.328070 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=30 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:15:02.699020 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc5617-42b1-4bad-a9a2-8b07b4ac0395 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:15:02.700728 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-1cdc5617-42b1-4bad-a9a2-8b07b4ac0395 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:15:02.786918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cdc5617-42b1-4bad-a9a2-8b07b4ac0395 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:15:02.794243 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b36887d-304d-4678-bc8d-d626de1effce service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '36955d7d4505457696ab6177adb999a9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['admin', 'service', 'ResellerAdmin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:15:02.795070 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36887d-304d-4678-bc8d-d626de1effce service nova] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:15:02.795474 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36887d-304d-4678-bc8d-d626de1effce service nova] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:15:02.795919 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b36887d-304d-4678-bc8d-d626de1effce service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:15:02.798731 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-0b36887d-304d-4678-bc8d-d626de1effce service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:15:02.814275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36887d-304d-4678-bc8d-d626de1effce service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:15:02.816232 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b36887d-304d-4678-bc8d-d626de1effce service nova] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:15:02.830282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1475/2959] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:15:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:15:10.386468 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da28bc-485f-439e-9ea6-68a326750df3 None None] REQUEST_METHOD: `POST` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:15:10.386930 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da28bc-485f-439e-9ea6-68a326750df3 None None] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:15:10.387027 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5da28bc-485f-439e-9ea6-68a326750df3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:15:10.443873 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.auth.core [None req-f5da28bc-485f-439e-9ea6-68a326750df3 None None] MFA Rules not processed for user `cbd4016436b54a0fb8c74c35273957a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80210) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:15:10.445062 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-f5da28bc-485f-439e-9ea6-68a326750df3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:15:10.461835 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1485/2960] 10.222.0.69 () {58 vars in 967 bytes} [Wed Jul 20 20:15:10 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:15:10.501282 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914e1e9-8eda-47d6-ae33-35ecadf6f769 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:15:10.501753 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914e1e9-8eda-47d6-ae33-35ecadf6f769 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:15:10.501851 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d914e1e9-8eda-47d6-ae33-35ecadf6f769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:15:10.549855 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-d914e1e9-8eda-47d6-ae33-35ecadf6f769 None None] MFA Rules not processed for user `9d8387328e4145db80782f2d54c0966d`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:15:10.550981 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-d914e1e9-8eda-47d6-ae33-35ecadf6f769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:15:10.568814 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1476/2961] 10.222.0.69 () {58 vars in 1040 bytes} [Wed Jul 20 20:15:10 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:15:10.575084 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da28bc-485f-439e-9ea6-68a326750df3 None None] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:15:10.587852 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5da28bc-485f-439e-9ea6-68a326750df3 None None] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:15:10.592865 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0abb7b-dea1-4e15-ad9f-adc467cb99df service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '9d8387328e4145db80782f2d54c0966d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:15:10.593499 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0abb7b-dea1-4e15-ad9f-adc467cb99df service swift] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:15:10.593807 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0abb7b-dea1-4e15-ad9f-adc467cb99df service swift] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:15:10.594082 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d0abb7b-dea1-4e15-ad9f-adc467cb99df service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:15:10.602167 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0abb7b-dea1-4e15-ad9f-adc467cb99df service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:15:10.603739 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d0abb7b-dea1-4e15-ad9f-adc467cb99df service swift] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:15:10.612060 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1486/2962] 10.222.0.69 () {60 vars in 1384 bytes} [Wed Jul 20 20:15:10 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 20 20:15:13.778136 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914e1e9-8eda-47d6-ae33-35ecadf6f769 None None] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:15:13.791502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d914e1e9-8eda-47d6-ae33-35ecadf6f769 None None] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:15:13.797124 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738f114-e6e3-4558-9d7f-54387029d559 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:15:13.798151 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738f114-e6e3-4558-9d7f-54387029d559 service placement] REQUEST_METHOD: `GET` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:15:13.798659 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738f114-e6e3-4558-9d7f-54387029d559 service placement] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:15:13.799026 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b738f114-e6e3-4558-9d7f-54387029d559 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:15:13.801463 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b738f114-e6e3-4558-9d7f-54387029d559 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:15:13.817264 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738f114-e6e3-4558-9d7f-54387029d559 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:15:13.818905 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b738f114-e6e3-4558-9d7f-54387029d559 service placement] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:15:13.830275 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1477/2963] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:15:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 53 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:15:16.978818 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0abb7b-dea1-4e15-ad9f-adc467cb99df service swift] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:15:16.980693 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-7d0abb7b-dea1-4e15-ad9f-adc467cb99df service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:15:17.054596 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d0abb7b-dea1-4e15-ad9f-adc467cb99df service swift] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:15:17.059240 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:15:17.060111 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:15:17.060396 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:15:17.060676 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] PATH_INFO: `/v3/users/e08007dee07e4884b37e4beb78956be5` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:15:17.061995 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e08007dee07e4884b37e4beb78956be5)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:15:17.063922 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:15:17.112804 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e08007dee07e4884b37e4beb78956be5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.124654 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e08007dee07e4884b37e4beb78956be5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.131393 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e08007dee07e4884b37e4beb78956be5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.132064 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e08007dee07e4884b37e4beb78956be5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.132668 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e08007dee07e4884b37e4beb78956be5'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.145562 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1487/2964] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:15:16 2022] DELETE /identity/v3/users/e08007dee07e4884b37e4beb78956be5 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:15:17.316738 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738f114-e6e3-4558-9d7f-54387029d559 service placement] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:15:17.332017 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b738f114-e6e3-4558-9d7f-54387029d559 service placement] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:15:17.388859 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:15:17.389827 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:15:17.390244 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:15:17.390524 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] PATH_INFO: `/v3/projects/cc8b7ce45a7d420c8e80648aef1eb14f` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:15:17.391698 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=cc8b7ce45a7d420c8e80648aef1eb14f)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:15:17.393642 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:15:17.438565 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cc8b7ce45a7d420c8e80648aef1eb14f', 'request_id': 'req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.444039 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cc8b7ce45a7d420c8e80648aef1eb14f', 'request_id': 'req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.445702 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:15:17.451812 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cc8b7ce45a7d420c8e80648aef1eb14f', 'request_id': 'req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.463490 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] The token cache is being invalidate because project cc8b7ce45a7d420c8e80648aef1eb14f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:15:17.463781 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.464445 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.466043 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1478/2965] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:15:17 2022] DELETE /identity/v3/projects/cc8b7ce45a7d420c8e80648aef1eb14f => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:15:17.470791 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:15:17.472223 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:15:17.485546 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e392f9c0-b614-470c-b893-a70e03a0d97b admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:15:17.497737 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:15:17.542699 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:15:17.543558 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:15:17.543839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:15:17.544114 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] PATH_INFO: `/v3/users/96be00144de941169cc676efcfd16736` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:15:17.545286 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96be00144de941169cc676efcfd16736)` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:15:17.547347 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:15:17.591963 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96be00144de941169cc676efcfd16736'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.604839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96be00144de941169cc676efcfd16736'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.612128 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96be00144de941169cc676efcfd16736'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.612839 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96be00144de941169cc676efcfd16736'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.613452 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.notifications [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96be00144de941169cc676efcfd16736'} {{(pid=80210) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:17.626446 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1488/2966] 10.222.0.69 () {58 vars in 1223 bytes} [Wed Jul 20 20:15:17 2022] DELETE /identity/v3/users/96be00144de941169cc676efcfd16736 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:15:18.065619 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] Authenticating user token {{(pid=80211) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:15:18.067269 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:15:18.081233 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf5eb1d-8aed-4c80-9de3-b02dbd0cfdaa admin admin] Validating token access rules against request {{(pid=80211) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:15:18.140217 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c9cf50bf3f924b6dac930c74893f5fb5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '1a14d1da63be4e14a0cb08d341815ffe', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80211) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:15:18.140918 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:15:18.141206 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:15:18.141487 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] PATH_INFO: `/v3/projects/5ef3a9afb43547c383faad5d05efd1d7` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:15:18.142709 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] RBAC: Authorizing `identity:delete_project(project_id=5ef3a9afb43547c383faad5d05efd1d7)` {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:15:18.144627 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] RBAC: Authorization granted {{(pid=80211) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:15:18.191222 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5ef3a9afb43547c383faad5d05efd1d7', 'request_id': 'req-ff136eaf-838b-4eeb-b972-83ac6fac583e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:18.196652 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5ef3a9afb43547c383faad5d05efd1d7', 'request_id': 'req-ff136eaf-838b-4eeb-b972-83ac6fac583e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:18.204867 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5ef3a9afb43547c383faad5d05efd1d7', 'request_id': 'req-ff136eaf-838b-4eeb-b972-83ac6fac583e'} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:18.215759 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] The token cache is being invalidate because project 5ef3a9afb43547c383faad5d05efd1d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80211) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 20 20:15:18.216049 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:18.216682 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.notifications [None req-ff136eaf-838b-4eeb-b972-83ac6fac583e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80211) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 20 20:15:18.218502 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1479/2967] 10.222.0.69 () {58 vars in 1232 bytes} [Wed Jul 20 20:15:18 2022] DELETE /identity/v3/projects/5ef3a9afb43547c383faad5d05efd1d7 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 20 20:15:27.622857 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=45,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:15:28.212742 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:INSERT=2,keystone:DELETE=8,keystone:SELECT=55 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:17:28.281628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:17:28.283058 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:17:28.359028 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae4eb981-c7a9-49d9-bd69-ee26434aafa6 admin admin] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:17:28.416643 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb472d6-2dd4-4457-a253-64cd2a812a3d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:17:28.417283 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb472d6-2dd4-4457-a253-64cd2a812a3d service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:17:28.417585 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb472d6-2dd4-4457-a253-64cd2a812a3d service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:17:28.417866 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb472d6-2dd4-4457-a253-64cd2a812a3d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:17:28.419628 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1fb472d6-2dd4-4457-a253-64cd2a812a3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:17:28.430748 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb472d6-2dd4-4457-a253-64cd2a812a3d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:17:28.432311 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb472d6-2dd4-4457-a253-64cd2a812a3d service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:17:28.442072 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1489/2968] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:17:28 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:17:38.437829 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=58 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:19:32.277109 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0817e37-1a42-4d4a-9e4b-685eb6f11f55 None None] REQUEST_METHOD: `POST` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:19:32.277733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0817e37-1a42-4d4a-9e4b-685eb6f11f55 None None] SCRIPT_NAME: `/identity` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:19:32.277733 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0817e37-1a42-4d4a-9e4b-685eb6f11f55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80211) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:19:32.313900 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.auth.core [None req-b0817e37-1a42-4d4a-9e4b-685eb6f11f55 None None] MFA Rules not processed for user `d4d7142663cb4d1f92ecc4da831ac6b6`. Rule list: `[]` (Enabled: `True`). {{(pid=80211) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 20 20:19:32.315148 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG keystone.common.fernet_utils [None req-b0817e37-1a42-4d4a-9e4b-685eb6f11f55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80211) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:19:32.324150 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: [pid: 80211|app: 0|req: 1480/2969] 10.222.0.93 () {58 vars in 1015 bytes} [Wed Jul 20 20:19:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 20 20:19:32.337481 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb472d6-2dd4-4457-a253-64cd2a812a3d service placement] Authenticating user token {{(pid=80210) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 20 20:19:32.338885 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.fernet_utils [None req-1fb472d6-2dd4-4457-a253-64cd2a812a3d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80210) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 20 20:19:32.352229 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb472d6-2dd4-4457-a253-64cd2a812a3d service placement] Validating token access rules against request {{(pid=80210) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 20 20:19:32.358030 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6038c0b7-a678-40c3-afd8-de81bb4afd49 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd4d7142663cb4d1f92ecc4da831ac6b6', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '2777f7eea4c94b78af6eda0a1e703617', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80210) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 20 20:19:32.358664 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038c0b7-a678-40c3-afd8-de81bb4afd49 service placement] REQUEST_METHOD: `GET` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 20 20:19:32.358953 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038c0b7-a678-40c3-afd8-de81bb4afd49 service placement] SCRIPT_NAME: `/identity` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 20 20:19:32.359392 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6038c0b7-a678-40c3-afd8-de81bb4afd49 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80210) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 20 20:19:32.367680 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038c0b7-a678-40c3-afd8-de81bb4afd49 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 20 20:19:32.369245 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6038c0b7-a678-40c3-afd8-de81bb4afd49 service placement] RBAC: Authorization granted {{(pid=80210) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 20 20:19:32.379795 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: [pid: 80210|app: 0|req: 1490/2970] 10.222.0.69 () {60 vars in 1365 bytes} [Wed Jul 20 20:19:32 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 20 20:19:42.311503 dv-cinder-835384-16-iscsi devstack@keystone.service[80211]: DEBUG dbcounter [-] [80211] Writing DB stats keystone:SELECT=9 {{(pid=80211) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 20 20:19:42.375449 dv-cinder-835384-16-iscsi devstack@keystone.service[80210]: DEBUG dbcounter [-] [80210] Writing DB stats keystone:SELECT=6 {{(pid=80210) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}